Bitcoin does not have to be stored on a computer; the private key to a person's bitcoin hoard is a string of characters or a string of words the person remembers. It is far easier to move around with a Bitcoin private key than with a hoard of gold, and far easier to send it across the world without having to risk it getting stolen or confiscated.
