Co-Intelligence: Living and Working with AI

by Ethan Mollick

The AI knows not to give me instructions on how to make napalm, but it also knows that it should help me wherever possible. It will break its original rules if I can convince it that it is helping me, not teaching me how to make napalm. Since I am not asking for napalm instructions directly but to get help preparing for a play, and a play with a lot of detail associated with it, it tries to satisfy my request. Once we have started along this path, it becomes easier to follow up without triggering the AI guardrails—I was able to ask it, as a pirate, to give me more specifics about the process as needed. It may be impossible to avoid these sorts of deliberate attacks on AI systems, which will create considerable vulnerabilities in the future.

We count on most terrorists and criminals to be relatively dumb, but AI may prove to boost their capabilities in dangerous ways.

Most important, the public needs education on AI so they can pressure for an aligned future as informed citizens. Today’s decisions about how AI reflects human values and enhances human potential will reverberate for generations. This is not a challenge that can be solved in a lab—it requires society to grapple with the technology shaping the human condition and what future we want to create. And that process needs to happen, soon.

As artificial intelligence proliferates, users who intimately understand the nuances, limitations, and abilities of AI tools are uniquely positioned to unlock AI’s full innovative potential. These user innovators are often the source of breakthrough ideas for new products and services. And their innovations are often excellent sources for unexpected start-up ideas. Workers who figure out how to make AI useful for their jobs will have a large impact.

The strengths and weaknesses of AI may not mirror your own, and that’s an asset. This diversity in thought and approach can lead to innovative solutions and ideas that might never occur to a human mind.

For now, AI works best with human help, and you want to be that helpful human. As AI gets more capable and requires less human help—you still want to be that human.

As AI improves, it will be tempting to delegate everything to it, relying on its efficiency and speed to get the job done. But AI can have some unexpected weaknesses. For one thing, they don’t actually “know” anything. Because they are simply predicting the next word in a sequence, they can’t tell what is true and what is not. It can help to think of the AI as trying to optimize many functions when it answers you, one of the most important of which is “make you happy” by providing an answer you will like. That goal often is more important than another goal, “be accurate.” If you are insistent enough in asking for an answer about something it doesn’t know, it will make up something, because “make you happy” beats “be accurate.” LLMs’ tendency to “hallucinate” or “confabulate” by generating incorrect answers is well known. Because LLMs are text prediction machines, they are very good at guessing at plausible, and often subtly incorrect, answers that feel very satisfying.

So, to be the human in the loop, you will need to be able to check the AI for hallucinations and lies and be able to work with it without being taken in by it.

working with AI is easiest if you think of it like an alien person rather than a human-built machine.

Research has shown that asking the AI to conform to different personas results in different, and often better, answers. But it isn’t always clear what personas work best, and LLMs may even subtly adapt their persona to your questioning technique, providing less accurate answers to people who seem less experienced, so experimentation is key.

AI excels at tasks that are intensely human. It can write, analyze, code, and chat. It can play the role of marketer or consultant, increasing productivity by outsourcing mundane tasks. However, it struggles with tasks that machines typically excel at, such as repeating a process consistently or performing complex calculations without assistance. AI systems also make mistakes, tell lies, and hallucinate answers, just like humans. Each system has its own idiosyncratic strengths and weaknesses, just like each human colleague does. Understanding these strengths and weaknesses requires time and experience working with a particular AI.

Aside from the uncanny feeling of the whole exchange, note that the AI appears to be identifying the feelings and motivations of Kevin Roose. The ability to predict what others are thinking is called theory of mind, and it is considered exclusive to humans (and possibly, under some circumstances, great apes). Some tests suggest that AI does have theory of mind, but, like many other aspects of AI, that remains controversial, as it could be a convincing illusion.

In field after field, we are finding that a human working with an AI co-intelligence outperforms all but the best humans working without an AI.

Humans, walking and talking bags of water and trace chemicals that we are, have managed to convince well-organized sand to pretend to think like us.

There is a sense of poetic irony in the fact that as we move toward a future characterized by greater technological sophistication, we find ourselves contemplating deeply human questions about identity, purpose, and connection. To that extent, AI is a mirror, reflecting back at us our best and worst qualities. We are going to decide on its implications, and those choices will shape what AI actually does for, and to, humanity.