The years of deprivation in several forms (a missing father, possible humiliation in accepting charity from relatives, and the subsequent deaths of his brothers and mother leaving him orphaned at the age of thirteen, to mention only the most obvious) probably provoked this hostility.
