He had married the love of his life, Rachel Donelson Robards, before she was divorced from her first husband. The scandal of his marriage stayed with him through the decades, and he believed that the stress of the charges of adultery and bigamy ultimately killed her.
