Lena! he cried. A violence of grief and self-recrimination blazed up in him.
Covenant claims to be filled with grief and self-recrimination over his rape of Lena. An experience he first luxuriated in and then fled with nary a thought for Lena's welfare ... if he is so upset, why does he not seek justice for Lena by submitting himself to just punishment.
Of course, Covenant must not be duly punished for rape, or else the subtle endorsement of rape culture in this series would be denied.