In earlier times in our evolution, there was no real way to interact with others at a distance any farther than that allowed by hands, feet, or possibly a stick. That distance of interaction was salient and consequential, and this is what our emotional reaction reflects.
