Nothing that a client does can change this, with one caveat: a privileged client can invoke the private constructor reflectively (Item 65) with the aid of the AccessibleObject.setAccessible method. If you need to defend against this attack, modify the constructor to make it throw an exception if it’s asked to create a second instance.
Don't mistake this for a security measure; this probably isn't enough to defend from adversarial code running within your JVM. In other words, there's a very narrow window in which this mechanism makes sense to employ, and I doubt it'll ever be the right tool for most of you!