“The powers not delegated to the United States by the Constitution nor prohibited by it to the States are reserved to the States respectively, or to the people.”
This may showcase the reasoning behind doing away with segregation, in that segregation violates a right reserved to the people, rather than to the States.

