Black Box Thinking: Why Some People Never Learn from Their Mistakes - But Some Do

by Matthew Syed

“You can have the best procedures in the world but they won’t work unless you change attitudes toward error.”

first, you need to have the right kind of system—one that harnesses errors as a means of driving progress; and second, you need a mindset that enables such a system to flourish.

Cognitive dissonance occurs when mistakes are too threatening to admit to, so they are reframed or ignored.

internal fear of failure: how we struggle to admit mistakes to ourselves.

We will see that all systems that learn from failure have a distinctive structure, one that can be found in many places, including the natural world, artificial intelligence, and science.

They all share an essential pattern: an adaptive process driven by the detection and response to failure.

Cumulative selection works, then, if there is some form of “memory”: i.e., if the results of one selection test are fed into the next, and into the next, and so on.

They were manned by intelligent planners who decided how much grain to produce, how much iron to mine, and who used complicated calculations to determine the optimal solutions.

their ideas, however enlightened, were not tested rapidly enough—and so had little opportunity to be reformed in the light of failure.

When companies go under, other entrepreneurs learn from these mistakes, the system creates new ideas, and consumers ultimately benefit.

make judicious use of tests, challenge their own assumptions, and wield the lessons to guide strategy. It is a mix of top-down reasoning (as per the mathematicians) and bottom-up iteration (as per the biologists); the fusing of the knowledge they already have with the knowledge that can be gained by revealing its inevitable flaws.

they owed nothing to science; they were empirical developments based on the trial, error and experimentation of skilled craftsmen who were trying to improve the productivity, and so the profits, of their factories.

Trial and error inspired the technology, which in turn inspired the theory. This is the linear model in reverse.

Ultimately, technological progress is a complex interplay between theoretical and practical knowledge, each informing the other in an upward spiral*.

Moreover, although it had been preprogrammed with a great deal of chess knowledge, it couldn’t learn from its own mistakes as it played the games.

This gave Kasparov a fighting chance, because he had something the computer largely lacked: practical knowledge developed through trial and error.

that there is a profound obstacle to testing, a barrier that prevents many of us from harnessing the upsides of the evolutionary process.

we are hardwired to think that the world is simpler than it really is. And if the world is simple, why bother to conduct tests?

narrative fallacy.

our propensity to create stories about what we see after the event.

journalists didn’t even notice that they had attempted to explain contradictory effects with the same underlying cause. That is the power of the narrative fallacy.

when we are misled into regarding the world as simpler than it really is, we not only resist testing our top-down strategies and assumptions, we also become more defensive when they are challenged by our peers or by the data.

It is difficult to speak up about concerns, because powerful egos come into play. The consequence is simple: the system doesn’t evolve.

It is the problem of valuing top-down over bottom-up. The second fallacy is the fear of failure.

The notion of getting into the trial and error process early informs one of the most elegant ideas to have emerged from the high-tech revolution: the lean start-up.

Houston had an insight. He realized that the MVP doesn’t need to be a working prototype at all.

All it has to do is mimic the essential features of the final product.

Provided it is sufficiently repre...

demonstrate whether consumers really want to buy it and thus kick-start the pro...

So Houston created a video that showed how the product woul...

“This is decidedly different from market research,” Ries writes. “If Swinmurn had relied on existing market research or conducted a survey, it could have asked what customers thought they wanted. By building a product instead, albeit a simple one, the company learned much more.”

bringing early adopters into the design process itself.

Agile scrum development and the fail-fast approach are just two of these.

as soon as it comes into contact with the real world—this is when you start to discover the flaws in the blueprint.

fusing what we already know, and what we can still learn.

identifying where one’s strategy is going wrong, and evolving.

449 failures to create a single success.

Has your company failed that often, and been honest enough to admit it?

In the absence of data, narrative is the best we have. But this is why we need to conduct tests, to challenge our hunches, and the narrative fallacies upon which they are often based.

Instead of trusting in narrative, we should be wielding the power of the evolutionary mechanism.

How can something be a failure when the statistics seem to show that it is a success?

we will examine one of the most important scientific innovations of the last two hundred years,

heart of the closed-loop phenomenon—and how to overcome it. The randomized control trial.

there is a third way that closed loops are sustained over time: through skewed interpretation.

“counterfactual.” It is all the things that could have happened but which in everyday experience we never observe because we did something else.

We can speculate on what would have happened, and we can make decent guesses. But we don’t really know.

redesign your company website and that sales subsequently increase.

you to believe that the redesign of the website caused the boost in sales.

how can you b...

Perhaps sales went up not because of the new website, but because a rival went bust, or interest rates went down, or because it was a rainy month and more people shopped online.

Complex systems require controls, ways to test under conditions where many factors are outside your control.