Black Box Thinking: Why Some People Never Learn from Their Mistakes - But Some Do

by Matthew Syed

The more we have riding on our judgments, the more we are likely to manipulate any new evidence that calls them into question.

Judges typically write one-line orders, not official opinions, meaning that they don’t analyze what went wrong. Neither does anyone else.

physicians cope with their errors through a process of denial. They “block mistakes from entering conscious thought” and “narrow the definition of a mistake so that they effectively disappear, or are seen as inconsequential.”*

It hints at the suspicion that the intellectual energy of some of the world’s most formidable thinkers is directed, not at creating new, richer, more explanatory theories, but at coming up with ever-more-tortuous rationalizations as to why they were right all along.

The desire for perfection rests upon two fallacies. The first resides in the miscalculation that you can create the optimal solution sitting in a bedroom or ivory tower and thinking things through rather than getting out into the real world and testing assumptions, thus finding their flaws. It is the problem of valuing top-down over bottom-up.

You are so worried about messing up that you never even get on the field of play.

Nick Swinmurn, another technology entrepreneur, created a rather different MVP. He reckoned the world needed a website in order to purchase a stylish collection of shoes.

In 2009 Swinmurn sold his company, Zappos, to Amazon for $1.2 billion.

Only when they had learned from direct feedback and early failures did they roll out big, across the nation, with disciplined consistency.

To leverage the power of failure, you have to be resilient and open. In other words, you have to have the right mindset as well as the right system.

If you run away from mistakes, you won’t get anywhere. “It is a very grueling experience,” he said. “One day you are on top of the world . . . the next day there is a huge bug and the site is down and you are tearing your hair out . . . And guess what: that is still true today.”

In 2014 Houston’s company was valued at just over $10 billion. It is called Dropbox.

Instead of trusting in narrative, we should be wielding the power of the evolutionary mechanism.

of the seventeen youngsters, sixteen were still going straight three months later.

wider program had had a dramatic impact on reoffending rates.

Participating communities report that 80 to 90 percent of the kids that they send to Rahway go straight after leaving this stage. That is an amazing success story. And it is unequalled by traditional rehabilitation methods.

Rigorous testing would later prove that the kids who were taken on prison visits were more likely to commit offenses in the future, not less—as we shall see.

more appropriate name for Scared Straight might have been Scared Crooked. It was an unequivocal failure. It damaged kids in a number of ways.

We don’t observe what would have happened if we had not gotten married. Or see what would have happened if we had taken a different job. We can speculate on what would have happened, and we can make decent guesses. But we don’t really know. This may seem like a trivial point, but the implications are profound.

by comparing the two groups, it is possible to see that, far from saving people as medieval doctors sincerely believed, bloodletting, on average, kills them.

This fact would have been invisible without the control group.* And this is why, as we noted in chapter 1, bloodletting survived as a recognized treatment until the nineteenth century.

“It is about marginal gains,”

“The approach comes from the idea that if you break down a big goal into small parts, and then improve on each of them, you will deliver a huge increase when you put them all together.”

The world is too complex to figure everything out from your armchair. The only way to be sure is to go out and test your ideas and programs, and to realize that you will often be wrong. But that is not a bad thing. It leads to progress.

At what point does an exercise in incremental cost-cutting start to impact on the bottom line?

I would much rather have clear answers than to delude myself that I have the ‘right’ answers.”

Mercedes takes this one step further. They use the first test not to improve the strategy, but to create richer feedback. Only when they have a deeper understanding of all the relevant data do they start to iterate.

One of the pit stops I witnessed was completed in an astonishing 1.95 seconds.*

The secret to modern F1 is not really to do with big ticket items; it is about hundreds of thousands of small items, optimized to the nth degree.

Success is about creating the most effective optimization loop.

drive the car, but I have an incredible operation behind me,” Hamilton said. Vowles added: “We will enjoy tonight, but tomorrow we will feed what we learned today into the next stage of the optimization loop.”

“F1 is an unusual environment because you have incredibly intelligent people driven by the desire to win,” he said. “The ambition spurs rapid innovation. Things from just two years ago seem antique. Standing still is tantamount to extinction.”

Harrah’s Casino Group is symbolic of the quiet revolution that has been taking place. The brand, which operates casinos and resorts across America, reportedly has three golden rules for staff: “Don’t harass women, don’t steal, and you’ve got to have a control group.” • • •

it turns out that epiphanies often happen when we are in one of two types of environment. The first is when we are switching off: having a shower, going for a walk, sipping a cold beer, daydreaming. When we are too focused, when we are thinking too literally, we can’t spot the obscure associations that are so important to creativity. We have to take a step back for the “associative state” to emerge. As the poet Julia Cameron put it: “I learned to get out of the way and let that creative force work through me.”

The other type of environment where creative moments often happen, as we have seen, is when we are being sparked by the dissent of others. When Kevin Dunbar, a psychologist at McGill University, went to look at how scientific breakthroughs actually happen, for example (he took cameras into four molecular biology labs and recorded pretty much everything that took place), he assumed that it would involve scientists beavering away in isolated contemplation. In fact, the breakthroughs happened at lab meetings, where groups of researchers would gather around a desk to talk through their work. Why here? Because they were forced to respond to challenges and critiques from their fellow researchers. They were jarred into seeing new associations.

this helps to explain why cities are so creative, why atriums are important; in fact why any environment that allows disparate people, and therefore ideas, to bump into each other, is so conducive. They facilitate the association of diverse ideas, and bring people face-to-face with dissent and criticism. All help to ignite creativity.

To spark the imagination and take our insights to their fullest expression, we should not insulate ourselves from failure; rather, we should engage with it.

Today education is conceived as providing young people with a body of knowledge. Students are rewarded when they apply this knowledge correctly. Failures are punished.

We learn not just by being correct, but also by being wrong. It is when we fail that we learn new things, push the boundaries, and become more creative. Nobody had a new insight by regurgitating information, however sophisticated.

We live in a world of experts. There is nothing particularly wrong with that. The expertise we have developed is crucial for all of us. But when we are trying to solve new problems, in business or technology, we need to reach beyond our current expertise. We do not want to know how to app...

Dyson advocates that we provide children with the tools they need not just to answer questions, but to ask questions. “The problem with academia is that it is about being good at remembering things like chemical formulae and theories, because that is what you have to regurgitate. But children are not allowed to...

My strategy has always been: be wrong as fast as we can . . . which basically means, we’re gonna screw up, let’s just admit that. Let’s not be afraid of that. But let’s do it as fast as we can so we can get to the answer.

We will see that blame is, in many respects, a subversion of the narrative fallacy: an oversimplification driven by biases in the human brain. We will also see that it has subtle but measurable consequences, undermining our capacity to learn.

we have to engage with the complexity of the world if we are to learn from it; we have to resist the hardwired tendency to blame instantly, and look deeper into the factors surrounding error if we are going to figure out what really happened and thus create a culture based upon openness and honesty rather than defensiveness and back-covering.

if you don’t know what went wrong, how can you put things right?

if professionals think they are going to be blamed for honest mistakes, why would they be open about them? If they do not trust their managers to take the trouble to see what really happened, why would they report what is going wrong, and how can the system adapt?

in a complex world this analysis flips on its head. In the worlds of business, politics, aviation, and health care, people often make mistakes for subtle, situational reasons. The problem is often not a lack of focus, it is a consequence of complexity. Increasing punishment, in this context, doesn’t reduce mistakes, it reduces openness. It drives the mistakes underground. The more unfair the culture, the greater the punishment for honest mistakes and the faster the rush to judgment, the deeper this information is buried. This means that lessons are not learned, so the same mistakes are made again and again, leading to more punitive punishment, and even deeper concealment and back-covering.

All this sounds plausible, but now think of the cultural ramifications. The board thought they had sent a strong signal that they were tough on mistakes; they had, in fact, sent a chilling message to their staff. If you fail, we will blame you. If you mess up, you will be scapegoated. They had told their staff, with an eloquence that no memo could ever match: “Act defensively, cover your backs, and cover up the precious information that we need to flourish.”

It is noteworthy that even experienced aviation investigators fall prey to the fundamental attribution error. When they are first confronted with an accident, the sense-making part of the brain is already creating explanations before the black box has been discovered. This is why studies have shown that their first instinct is almost always (around 90 percent of the time) to blame “operator error.”

Forward-looking accountability is nothing more and nothing less than learning from failure.