Black Box Thinking: Growth Mindset and the Secrets of High Performance

by Matthew Syed

It is not easy for a grieving family to insist on an investigation when the experts are telling them it is not necessary. But Martin Bromiley wouldn’t give up. Why? Because he had spent his entire professional life in an industry with a different – and unusual – attitude to failure. He is a pilot. He has flown for commercial airlines for more than twenty years. He has even lectured on system safety. He didn’t want the lessons from a botched operation to die along with his wife. So he asked questions. He wrote letters. And as he discovered more about the circumstances surrounding his wife’s death, he began to suspect that it wasn’t a one-off. He realised that the mistake may have had a ‘signature’, a subtle pattern which, if acted upon, could save future lives. The doctors in charge of the operation couldn’t have known this for a simple but devastating reason: historically, healthcare institutions have not routinely collected data on how accidents happen, and so cannot detect meaningful patterns, let alone learn from them. In aviation, on the other hand, pilots are generally open and honest about their own mistakes (crash-landings, near misses). The industry has powerful, independent bodies designed to investigate crashes. Failure is not regarded as an indictment of the specific pilot who messes up, but a precious learning opportunity for all pilots, all airlines and all regulators.

Instead of concealing failure, or skirting around it, aviation has a system where failure is data rich. In the event of an accident, investigators, who are independent of the airlines, the pilots union and the regulators, are given full rein to explore the wreckage and to interrogate all other evidence. Mistakes are not stigmatised, but regarded as learning opportunities.

This, then, is what we might call ‘black box thinking’.fn8 For organisations beyond aviation, it is not about creating a literal black box; rather, it is about the willingness and tenacity to investigate the lessons that often exist when we fail, but which we rarely exploit. It is about creating systems and cultures that enable organisations to learn from errors, rather than being threatened by them.

The first is that you have to take into account all the data, including the data you cannot immediately see, if you are going to learn from adverse incidents. But it also emphasises that learning from failure is not always easy, even in conceptual terms, let alone emotional terms. It takes careful thought and a willingness to pierce through the surface assumptions. Often, it means looking beyond the obvious data to glimpse the underlying lessons. This is not just true of learning in aviation, but in business, politics and beyond.

The irony is that Sullenberger, feted by presidents, might have made the precisely same mistake in the same circumstances. The fact that he didn’t, and emerged a hero, was for a simple but profound reason: the industry in which he operates had learned the lessons. It is both apt and revealing that Sullenberger, a modest and self-evidently decent man, has made precisely this point. In a television interview months after the miracle landing on the Hudson, he offered this beautiful gem of wisdom: Everything we know in aviation, every rule in the rule book, every procedure we have, we know because someone somewhere died . . . We have purchased at great cost, lessons literally bought with blood that we have to preserve as institutional knowledge and pass on to succeeding generations. We cannot have the moral failure of forgetting these lessons and have to relearn them.

How can experience be so valuable in some professions, but almost worthless in others? To see why, suppose that you are playing golf. You are out on the driving range, hitting balls towards a target. You are concentrating and every time you fire the ball wide, you adjust your technique in order to get it closer to where you want it to go. This is how practice happens in sport. It is a process of trial and error. But now suppose that instead of practising in daylight, you practise at night – in the pitch-black. In these circumstances, you could practise for ten years or ten thousand years without improving at all. How could you progress if you don’t have a clue where the ball has landed? With each shot, it could have gone long, short, left or right. Every shot has been swallowed by the night. You wouldn’t have any data to improve your accuracy.

Feedback, when delayed, is considerably less effective in improving intuitive judgement.

‘You can have the best procedures in the world but they won’t work unless you change attitudes towards error.’

When we are confronted with evidence that challenges our deeply held beliefs we are more likely to reframe the evidence than we are to alter our beliefs. We simply invent new reasons, new justifications, new explanations. Sometimes we ignore the evidence altogether.

The next stage was for the prosecutor to argue that the semen belonged to a different man who was not the murderer. In other words, the victim had had consensual sex with another man, but had subsequently been raped by the prisoner, who had used a condom.22 This is the domino effect of cognitive dissonance: the reframing process takes on a life of its own. The presence of an entirely new man, not mentioned at the initial trial, for whom there were no eyewitnesses, and who the victim often couldn’t remember having sex with, may seem like a desperate ploy to evade the evidence. But it has been used so often that it has been given a name by defence lawyers: ‘the unindicted coejaculator’. It is a term that usefully captures the power of cognitive dissonance.

Self-justification is more insidious. Lying to oneself destroys the very possibility of learning. How can one learn from failure if one has convinced oneself – through the endlessly subtle means of self-justification, narrative manipulation, and the wider psychological arsenal of dissonance-reduction – that a failure didn’t actually occur?

But this perspective does not encompass the full influence of cognitive dissonance. The problem is not just the external incentive structure, it is the internal one. It is the sheer difficulty that we have in admitting our mistakes even when we are incentivised to do so. To see this most clearly, consider the so-called disposition effect, a well-studied phenomenon in the field of behavioural finance. Say you have a portfolio of shares, some of which have lost money, and some of which have gained. Which are you likely to sell? And which are you likely to keep? A rational person should keep those shares most likely to appreciate in the future while selling those likely to depreciate. Indeed, this is what you must do if you are attempting to maximise your financial return. The stock market rewards those who buy low and sell high. But we are actually more likely to keep the shares that have lost money, regardless of their future prospects. Why? Because we hate to crystallise a loss. The moment a losing stock is sold, a paper loss becomes a real loss. It is unambiguous evidence that the decision to buy that stock in the first place was a mistake. This is why people hold on to losing stocks far too long, desperately hoping they will rebound.

The pattern is rarely uncovered unless subjects are willing to make mistakes – that is, to test numbers that violate their belief. Instead most people get stuck in a narrow and wrong hypothesis, as often happens in real life, such that their only way out is to make a mistake that turns out not to be a mistake after all. Sometimes, committing errors is not just the fastest way to the correct answer; it’s the only way.

Cognitive dissonance occurs when mistakes are too threatening to admit to, so they are reframed or ignored. This can be thought of as the internal fear of failure: how we struggle to admit mistakes to ourselves.

What the development of the nozzle reveals, above all, is the power of testing. Even though the biologists knew nothing about the physics of phase transition, they were able to develop an efficient nozzle by trialling lots of different ones, rejecting those that didn’t work and then varying the best nozzle in each generation.

it is pretty much impossible to come up with perfect code first time around. It is only when people are using the software, putting it under strain, that you see the bugs and deficiencies you could never have anticipated. By putting the code out there and subjecting it to trial and error you learn the insights that create progress. Why, he asked Vanier, would you try to answer every question before you have a single user?

In their book Art and Fear David Bayles and Ted Orland tell the story of a ceramics teacher who announced on the opening day of class that he was dividing the students into two groups. Half were told that they would be graded on quantity. On the final day of term, the teacher said he would come to class with some scales and weigh the pots they had made. They would get an ‘A’ for 50 lbs of pots, a ‘B’ for 40 lbs, and so on. The other half would be graded on quality. They just had to bring along their one, perfect pot. The results were emphatic: the works of highest quality were all produced by the group graded for quantity. As Bayles and Orland put it: ‘It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.’

You see this in politics, too. Politicians come up with theories (bordering on ideologies) about whether, say, wearing school uniform improves discipline. They talk to psychologists and debate the issue in high-level meetings. It is an elaborate, top-down waste of time. They end up with dead clay. They should conduct a test, see what works, and what doesn’t. They will fail more, but that is precisely why they will learn more.

‘It is about marginal gains,’ he said. ‘The approach comes from the idea that if you break down a big goal into small parts, and then improve on each of them, you will deliver a huge increase when you put them all together.’ It sounds simple, but as a philosophy, marginal gains has become one of the hottest concepts not just in sport, but beyond. It has formed the basis of business conferences, seminars and has even been debated in the armed forces. Many British sports now employ a director of marginal gains.

Undeterred, the economists started to think about the problem in a fresh way. They tried something completely new: a de-worming medication. This may seem like a curious way to improve education, but researchers were aware that these parasites stunt growth, cause children to feel lethargic, and lead to absenteeism. They disproportionately affect children in remote communities, just like those in Busia and Teso. This time the results were excellent. They vastly exceeded the expectations of the researchers. As Tim Harford put it: ‘The programme was a huge success, boosting children’s height, reducing re-infection rates, and also reducing absenteeism from school by a quarter. And it was cheap.’5 This was a marginal gain. It was just one programme in one small region. But by looking at education at this level of magnification, it was possible to see what really works, and what doesn’t. The economists had tested, failed and learned. They could now roll it out in other areas, while continuing to test, and iterate, and create yet more marginal gains.

In Corporate Creativity, the authors Alan Robinson and Sam Stern write of how Bob Crandall, the former chairman of American Airlines, removed a single olive from every salad, and in doing so saved $500,000 annually.8 Many seized on this as a marginal gain. But was it? After all, if removing an olive is a good idea, why not the lettuce too? At what point does an exercise in incremental cost-cutting start to impact on the bottom line?

The basic proposition of this book is that we have an allergic attitude to failure. We try to avoid it, cover it up and airbrush it from our lives. We have looked at cognitive dissonance, the careful use of euphemisms, anything to divorce us from the pain we feel when we are confronted with the realisation that we have underperformed.

Every error, every flaw, every failure, however small, is a marginal gain in disguise. This information is regarded not as a threat but as an opportunity.

Nathan’s Hot Dog Eating Contest, which takes place every 4 July in Coney Island, New York. The rules are straightforward: eat as many hot dogs and buns as you can in twelve minutes. You are allowed to drink anything you like, but you are not allowed to vomit significantly (a problem known in the sport as a ‘reversal of fortune’).

if our first reaction is to assume that the person closest to a mistake has been negligent or malign, then blame will flow freely and the anticipation of blame will cause people to cover up their mistakes. But if our first reaction is to regard error as a learning opportunity, then we will be motivated to investigate what really happened.

Overcoming the blame tendency is a defining issue in the corporate world. Ben Dattner, a psychologist and organisational consultant, tells of an experience when he was working at the Republic National Bank of New York. He noticed a piece of paper that a co-worker had stapled to his cubicle wall. It read: ‘The six phases of a project: 1. Enthusiasm 2. Disillusionment 3. Panic 4. Search for the guilty 5. Punishment of the innocent 6. Rewards for the uninvolved.’ Dattner writes: ‘I have yet to come across a more accurate description of how most dramas play out in our working lives.’

In one experiment by the psychologist Carol Dweck and a colleague, eleven- and twelve-year-olds were given eight easy tests, then four very difficult ones. As they worked, the two groups exhibited startlingly different responses.2 Here are the children in the Fixed Mindset grouping being described by Dweck: ‘Maybe the most striking thing about this group was how quickly they began to denigrate their abilities and blame their intelligence for the failures, saying things like “I guess I am not very smart”, “I never did have a good memory” and “I’m no good at things like this”.’ Two-thirds of them showed a clear deterioration in their strategies, and more than half of them lapsed into completely ineffective strategies.

And yet if young people think success happens instantly for the truly talented, why would they persevere? If they take up, say, the violin and are not immediately playing like a virtuoso, they are going to assume they don’t have what it takes – and so they will give up. In effect, the mistaken idea that success is an instant phenomenon destroys resilience.

A rational financial trader should keep shares that are most likely to appreciate in the future while selling those likely to depreciate. But traders are actually more likely to keep the shares that have lost money, regardless of future prospects. Why? Because they hate to crystallise a loss. This is why people hold on to losing stocks for far too long, desperately hoping they will rebound. Even professional stock pickers are vulnerable, holding losing stocks twice as long as winning stocks.

For centuries before the Greeks the entire weight of intellectual history was about preserving and defending established ideas: religious, practical and tribal. This defensive tendency, seemingly so universal in human history, has been a subject of speculation for anthropologists over many years. But the answer, surely, is that ancient tribes were trapped in a Fixed Mindset. They thought that the truth had been revealed by a god or god-like ancestor and did not feel any need to build new knowledge. New evidence was regarded not as an opportunity to learn fresh truths, but as a threat to the established worldview.

This is also true of developing expertise in sport. In sport, feedback is almost always instant and obvious. We know when we have hit a ball out of bounds in golf or mistimed a forehand in tennis. But enlightened training environments maximise the quantity and quality of feedback, thus increasing the speed of adaptation. Take football. Every time a player fails to control an incoming pass, he has learned something. Over time the central nervous system adapts, building more finesse and touch. But if a young player practises on a full-sized pitch, touching the ball infrequently, he will not improve very fast. On the other hand, if he practises on a smaller pitch, touching the ball frequently, his skill will improve more quickly.

A pre-mortem typically starts with the leader asking everyone in the team to imagine that the project has gone horribly wrong and to write down the reasons why on a piece of paper. He or she then asks everyone to read a single reason from the list, starting with the project manager, before going around the table again. Klein cites examples where issues have surfaced that would otherwise have remained buried. ‘In a session held at one Fortune 50-size company, an executive suggested that a billion-dollar environmental sustainability project had “failed” because interest waned when the CEO retired,’ he writes. ‘Another pinned the failure on a dilution of the business case after a government agency revised its policies.’15 The purpose of the pre-mortem is not to kill off plans, but to strengthen them. It is also very easy to conduct.

think about how rapidly we learn to steer a car. The feedback is instant and objective. It takes far longer to learn how to steer a ship, because there are long delays between actions and noticeable outcomes.