If food restriction is severe and lasts for a long time, our organs will actually shrink. But not all organ systems are hit equally hard. We know from careful studies of the bodies of victims starved to death in war and famine that the brain is spared. The spleen, on the other hand, shrinks dramatically.
