Flour, potatoes, and grains were brain-shrinking foods as well. Because of the way they spike blood sugar rapidly, all of these foods are essentially sugar at their most basic form. The only carbohydrate sources linked to a bigger brain were vegetables (not including potatoes) and whole fruits.