This answer contains spoilers… (view spoiler)[When a woman is raped, a lot of people blame her even now (they shouldn't obviously, but a lot do anyway). And back then, not only would a lot of people blame her, a lot of people would also consider her permanently "tainted" or "unclean" because he was black and she was white. Norwood might have thought that way. Or he might have blamed himself for leaving Fannie alone. Or he might have just been upset about the situation and needed to get away for a little while. (hide spoiler)]
Deborah This is a novel, and that situation was used to create the setting for other aspects of the story. I do think that at that time, especially in the south, the rape of a white woman by a black man might have been very devastating for a husband to accept, and did not seem surprised by that part of the story.