A little more on portal fantasy refugees

In the last entry, osprey_archer , sovay , and I got to talking (thread here ) about the background premise of Every Heart a Doorway--namely, that there could be some kind of special school for young people who can't readjust to life in our world after having an adventure in another world. I realized I had some questions that go beyond that particular novella.

(1) Where does the notion that a portal adventure leaves you messed up come from? Is it all down to the fact of not being able to return...