Black Box Thinking: Why Some People Never Learn from Their Mistakes - But Some Do

by Matthew Syed

trial and error, on its own, is sometimes insufficient to drive rapid progress. Why? Because social complexity can play havoc with the interpretation of observational feedback. Controlled trials, where practical and ethical, have the potential to boost learning by isolating causal relationships. And yet they are not a panacea. We have to be mindful of unintended consequences and the holistic context, which are sometimes neglected by those who perform RCTs.

If valid learning can be achieved through iteration at a fast pace and low cost, it is crazy to pass up the opportunity.

No one can possibly give us more service than by showing us what is wrong with what we think or do; and the bigger the fault, the bigger the improvement made possible by its revelation. The man who welcomes and acts on criticism will prize it almost above friendship: the man who fights it out of concern to maintain his position is clinging to non growth.

Another “failure based” technique, which has come into vogue in recent years, is the so-called pre-mortem. With this method a team is invited to consider why a plan has gone wrong before it has even been put into action. It is the ultimate “fail fast” technique. The idea is to encourage people to be open about their concerns, rather than hiding them out of fear of sounding negative. The pre-mortem is crucially different from considering what might go wrong. With a pre-mortem, the team is told, in effect, that “the patient is dead”: the project has failed; the objectives have not been met; the plans have bombed. Team members are then asked to generate plausible reasons why.

“prospective hindsight,” as it is called, increases the ability of people to correctly identify reasons for future outcomes by 30 percent.

*In many circumstances, task-focused behavior is actually an effective way of applying one’s effort. The problem is when this focus comes at the expense of the “bigger picture.” This is when excessive focus undermines performance and, in the case of aviation, safety.

*“Observational statistics” is a phrase that encompasses all the statistics drawn from looking at what happened. Randomized control trials are different because they encompass not merely what happened, but also construct a counterfactual for comparison.