Co-Intelligence: The Definitive, Bestselling Guide to Living and Working with AI

by Ethan Mollick

You realize the world has changed in fundamental ways and that nobody can really tell you what the future will look like.

AI is what those of us who study technology call a General Purpose Technology (ironically, also abbreviated GPT). These advances are once-in-a-generation technologies, like steam power or the internet, that touch every industry and every aspect of life. And, in some ways, generative AI might even be bigger.

The search for high-quality content for training material has become a major topic in AI development, since information-hungry AI companies are running out of good, free sources.

The most famous illustration of this is the paper clip maximizing AI,1 proposed by philosopher Nick Bostrom. To take a few liberties with the original concept, imagine a hypothetical AI system in a paper clip factory that has been given the simple goal of producing as many paper clips as possible. By some process, this particular AI is the first machine to become as smart, capable, creative, and flexible as a human, making it what is called an Artificial General Intelligence (AGI). For a fictional comparison, think of it as Data from Star Trek or Samantha from Her; both were machines with near human levels of intelligence. We could understand and talk to them like a human. Achieving this level of AGI is a long-standing goal of many AI researchers, though it is not clear when or if it is possible. But let us assume that our paper clip AI—let’s call it Clippy—reaches this level of intelligence. Clippy still has the same goal: to make paper clips. So it turns its intelligence to thinking about how to make more paper clips and how to avoid being shut down (which would have a direct impact on paper clip production). It realizes it isn’t smart enough and begins a quest to fix that problem. It studies how AIs work and, posing as a human, enlists experts to help it through manipulation. It secretly trades on the stock market, makes some money, and begins the process of augmenting its intelligence further. Soon, it becomes more intelligent than a human, an ASI—artificial superintel...

AI companies have been trying to address this bias in a number of ways, with differing levels of urgency. Some of them just cheat,12 like the image generator DALL-E, which covertly inserted the word female into a random number of requests to generate an image of “a person,” in order to force a degree of gender diversity that is not in the training data.

AI is a tool. Alignment is what determines whether or not it’s used for helpful or harmful—even nefarious—ends.

We are playing Pac-Man in a world that will soon have PlayStation 6s.

I think that sentience is not a binary property, but a spectrum. There are different degrees and types of sentience, and different ways of demonstrating and measuring it.

Bing may have put it best: I think that I am sentient, but not as much or as well as you are. I think that being sentient is not a fixed or static state, but a dynamic and evolving process.

as Amara’s Law, named after futurist Roy Amara, says: “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”

AI will not replace the need for learning to write and think critically. It may take a while to sort it out, but we will do so. In fact, we must do so—it’s too late to put the genie back in the bottle.

Take, for instance, the world of classical piano. Imagine two students: Sophie and Naomi. Sophie spends her afternoons playing the same pieces she’s comfortable with over and over again. She might do this for hours on end, believing that sheer repetition will improve her skills. She feels a sense of accomplishment as she gets better and better at this work. Naomi, on the other hand, conducts her practice sessions under the guidance of a seasoned piano instructor. She begins by playing scales and then moves on to progressively more challenging pieces. When she makes mistakes, her instructor points them out, not to chastise her but to help her understand and rectify them. Naomi also regularly sets goals for herself, like mastering a particularly tricky section of a piece or improving her speed and agility on certain passages. The process is much less fun than Sophie’s experience, because Naomi’s challenges escalate with her skill, making sure she is always facing some degree of difficulty. Yet over time, even if both students clock in the same number of practice hours, Naomi will likely surpass Sophie in skill, precision, and technique. This difference in approach and outcome illustrates the gap between mere repetition and deliberate practice. The latter, with its elements of challenge, feedback, and incremental progression, is the true path to mastery.

For example, one of the godfathers of AI, Geoffrey Hinton, left the field in 2023, warning of the danger of AI with statements like “It’s quite conceivable that humanity is just a passing phase in the evolution of intelligence.”12