two quotations on the brief dream of the human intellect

Neal Stephenson, today:

Speaking of the effects of technology on individuals and society as a whole, Marshall McLuhan wrote that every augmentation is also an amputation. I first heard that quote twenty years ago from a computer scientist at Stanford who was addressing a room full of colleagues—all highly educated, technically proficient, motivated experts who well understood the import of McLuhan’s warning and who probably thought about it often, as I have done, whenever they subsequently adopted some new labor-saving technology. Today, quite suddenly, billions of people have access to AI systems that provide augmentations, and inflict amputations, far more substantial than anything McLuhan could have imagined. This is the main thing I worry about currently as far as AI is concerned. I follow conversations among professional educators who all report the same phenomenon, which is that their students use ChatGPT for everything, and in consequence learn nothing. We may end up with at least one generation of people who are like the Eloi in H.G. Wells’s The Time Machine, in that they are mental weaklings utterly dependent on technologies that they don’t understand and that they could never rebuild from scratch were they to break down…. 

To me this seems like a downside of AI that is easy to understand, easy to measure, with immediate effects, that could be counteracted tomorrow through simple interventions such as requiring students to take examinations in supervised classrooms, writing answers out by hand on blank paper. We know this is possible because it’s how all examinations used to be taken. No new technology is required, nothing stands in the way of implementation other than institutional inertia, and, I’m afraid, the unwillingness of parents to see their children seriously challenged. In the scenario I mentioned before, where humans become part of a stable but competitive ecosystem populated by intelligences of various kinds, one thing we humans must do is become fit competitors ourselves. And when the competition is in the realm of intelligence, that means preserving and advancing our own intelligence by holding at arms length seductive augmentations in order to avoid suffering the amputations that are their price. 

H. G. Wells, from The Time Machine, 1895: 

I grieved to think how brief the dream of the human intellect had been. It had committed suicide. It had set itself steadfastly towards comfort and ease, a balanced society with security and permanency as its watchword, it had attained its hopes — to come to this at last. Once, life and property must have reached almost absolute safety. The rich had been assured of his wealth and comfort, the toiler assured of his life and work. No doubt in that perfect world there had been no unemployed problem, no social question left unsolved. And a great quiet had followed.

It is a law of nature we overlook, that intellectual versatility is the compensation for change, danger, and trouble. An animal perfectly in harmony with its environment is a perfect mechanism. Nature never appeals to intelligence until habit and instinct are useless. There is no intelligence where there is no change and no need of change. Only those animals partake of intelligence that have to meet a huge variety of needs and dangers.