The beautifully conserved circuitry of the basal ganglia, first emerging in the minuscule brain of early vertebrates and maintained for five hundred million years, seems to be the biological manifestation of Sutton’s actor-critic system. Sutton discovered a trick that evolution had already stumbled upon over five hundred million years ago.
