The protagonist lived in a foreign country a long time and came back to Japan. It's not that he lost someone close, but being away from a place you call home makes you feel like you've lost something very dear to you. As a person who lived in the states for a long time, I can relate to him. I understand when he said that it's difficult to even explain that sense of loss.
He likes to take care of all his plants and flowers at home and not socialize that much. I'm also the same way. But it was nice to see him get out of his shell and open up to someone.