In real life, scapegoats are usually singled out due to some combination of the following: they have extreme personalities or neurodiversity (such as autism) or physical abnormalities that make them noticeable; they’re on the margins of society in terms of status or markets (they are outside the system, like the Amish or people who have chosen to live off the grid);
