Differences in libido can be a source of shame for both sides, and claiming that sex shouldn’t matter at all or judging someone for wanting to leave is not helpful. If sex is important, let sex be important. It’s okay to leave and have sex with someone who wants to have sex with you. Just remember that leaving for sexual reasons does not mean the other person was wrong.