Alchemy: The Surprising Power of Ideas That Don't Make Sense

by Rory Sutherland

The need to appear scientific in our methodology may prevent us from considering other, less logical and more magical solutions, which can be cheap, fast-acting and effective. The mythical ‘butterfly effect’ does exist, but we don’t spend enough time butterfly hunting.

When you demand logic, you pay a hidden price: you destroy magic. And the modern world, oversupplied as it is with economists, technocrats, managers, analysts, spreadsheet-tweakers and algorithm designers, is becoming a more and more difficult place to practise magic – or even to experiment with it.

Most people spend their time at work trying to look intelligent, and for the last fifty years or more, people have tried to look intelligent by trying to look like scientists; if you ask someone to explain why something happened, they will generally give you a plausible-sounding answer that makes them seem intelligent, rational or scientific but that may or may not be the real answer. The problem here is that real life is not a conventional science – the tools which work so well when designing a Boeing 787, say, will not work so well when designing a customer experience or a tax programme. People are not nearly as pliable or predictable as carbon fibre or metal alloys, and we should not pretend that they are.

Adam Smith, the father of economics, identified this problem in the late eighteenth century,fn3 but it is a lesson which many economists have been ignoring ever since. If you want to look like a scientist, it pays to cultivate an air of certainty, but the problem with attachment to certainty is that it causes people completely to misrepresent the nature of the problem being examined, as if it were a simple physics problem rather than a psychological one. There is hence an ever-present temptation to pretend things are more ‘logical’ than they really are.

It’s important to remember that big data all comes from the same place – the past.

Being slightly bonkers can be a good negotiating strategy: being rational means you are predictable, and being predictable makes you weak. Hillary thinks like an economist, while Donald is a game theorist, and is able to achieve with one tweet what would take Clinton four years of congressional infighting. That’s alchemy; you may hate it, but it works.

If you are wholly predictable, people learn to hack you.

Do we look at things from a single perspective, where you do one thing to achieve another, or do we accept that complex things are rather different? In a designed system, such as a machine, one thing does serve one narrow purpose, but in an evolved or complex system, or in human behaviour, things can have multiple uses depending on the context within which they are viewed.

Taleb includes what might be the most interesting quotation on an individual’s politics I have ever read. Someonefn3 explains how, depending on context, he has entirely different political preferences: ‘At the federal level I am a Libertarian. At the state level, I am a Republican. At the town level, I am a Democrat. In my family I am a socialist. And with my dog I am a Marxist – from each according to his abilities, to each according to his needs.’

The problem that bedevils organisations once they reach a certain sizefn8 is that narrow, conventional logic is the natural mode of thinking for the risk-averse bureaucrat or executive. There is a simple reason for this: you can never be fired for being logical. If your reasoning is sound and unimaginative, even if you fail, it is unlikely you will attract much blame. It is much easier to be fired for being illogical than it is for being unimaginative.

There is the unambiguously ‘right’ answer, where certainty is achieved by limiting the number of data points considered. The downside of this is that, in the wrong context, it can be hopelessly wrong. Then there is the pretty good judgement of psycho-logic, which considers a far wider range of factors to arrive at a not-perfect-but-rarely-stupid conclusion.

The late David Ogilvy, one of the greats of the American advertising industry and the founder of the company I work for, apparently once said, ‘The trouble with market research is that people don’t think what they feel, they don’t say what they think, and they don’t do what they say.’fn6 Trivers and Kurzban explained the evolutionary science behind that conundrum: we simply don’t have access to our genuine motivations, because it is not in our interest to know.

Human self-deception makes our job difficult for another reason: no one wants to believe in its existence, and it is something which people seem only to accept at a shallow, theoretical level.fn8 People are much more comfortable attributing the success of a business to superior technology or better supply-chain management than to an unconscious, unspoken human desire. Perhaps that’s because we patently need a level of self-delusion to function as a social species.fn9 Imagine a world where we had no capacity for deception, and where people on dates directly asked prospective partners about their earning power and career prospects, without even pretending to be interested in their personalities. Where would we be then?

Using this as a metaphor, I would like to see the improvement we have enjoyed in food over the last three decades applied to other fields. It is only when we abandon a narrow logic and embrace an appreciation of psycho-logical value, that we can truly improve things. Once we are honest about the existence of unconscious motivations, we can broaden our possible solutions. It will free us to open up previously untried spaces for experimentation in resolving practical problems if we are able to discover what people really, really want,fn5 rather than a) what they say they want or b) what we think they should want.

My colleague Christopher Graves, who founded the Ogilvy Center for Behavioral Change in New York, calls this approach ‘asking the real why’. People may be accurate commentators on their emotional state, but the causes of that emotional state (in this case, uncertainty) are often a complete mystery to them. If the experiment works, and early indications are positive, we have performed a form of alchemy, using psycho-logic to conjure up value from nowhere.

Whether we use logic or psycho-logic depends on whether we want to solve the problem or to simply to be seen to be trying to solve the problem. Saving the world indirectly may not make you look like a hero; talking about the plight of polar bears makes one feel a good deal worthier than promoting the redesign of recycling bins, but the latter may be more effective. The self-regarding delusions of people in high-status professions lie behind much of this denial of unconscious motivation.

I was once walking down a suburban street in Wallingford, Pennsylvania. In American suburbia, there are no hedges to obscure the houses – a whitewashed fence about two feet high is all that marks property boundaries. So I was slightly alarmed when a large unleashed dog lurched towards me across one of the lawns, barking loudly. Clearly he was not going to have much difficulty clearing the little fence, after which he would be free to tear me to shreds. My companion, however, seemed unperturbed and sure enough, about two feet before the fence, the dog skidded to a halt on the lawn and continued its furious barking. As my friend knew, the dog was fitted with a collar that would detect the presence of a wire buried beneath the lawn boundary and administer an electric shock to the dog if it came too close. Although the fence was only two feet high, the dog was terrified to approach it. A similar constraint also applies to decision-making in business and government. There is a narrow and tightly limited area within which economic theory allows people to act. Once they reach the edges of that area, they freeze, rather like the dog. In some influential parts of business and government, economic logic has become a limiting creed rather than a methodological tool. As Sir Christopher Llewellyn Smith, the former director of CERN, remarked after he had been tasked with changing the patterns of energy consumption in Britain, ‘When I ask an economist, the answer always boils down to jus...

Strangely, as we have gained access to more information, data, processing power and better communications, we may also be losing the ability to see things in more than one way; the more data we have, the less room there is for things that can’t easily be used in computation. Far from reducing our problems, technology may have equipped us with a rational straitjacket that limits our freedom to solve them.

I would rather run a business with no mathematicians than with second-rate mathematicians. Remember that every time you average, add or multiply something, you are losing information. Remember also that a single rogue outlier can lead to an extraordinary distortion of reality – just as Bill Gates can walk into a football stadium and raise the average level of wealth of everyone in it by $1m.

In maths, 10 x 1 is always the same as 1 x 10, but in real life, it rarely is. You can trick ten people once, but it’s much harder to trick one person ten times.fn2

Everyone worries about declining social mobility, rising inequality and the hideous homogeneity of politicians, yet it is possible these have arisen from well-meaning attempts to make the world fairer. The quandary is that you can either create a fairer, more equitable society, with opportunities for all but where luck plays a significant role, or you can create a society which maintains the illusion of complete and non-random fairness, yet where opportunities are open to only a few – the problem is that when ‘the rules are the same for everyone’ the same boring bastards win every time. The idea that you should therefore try making your recruitment system less fair outrages people when I suggest it, but it is worth remembering that there is an inevitable trade-off between fairness and variety. By applying identical criteria to everyone in the name of fairness, you end up recruiting identical people.

Metrics, and especially averages, encourage you to focus on the middle of a market, but innovation happens at the extremes. You are more likely to come up with a good idea focusing on one outlier than on ten average users.

One of the other problems with a logically consistent system for hiring people is that ambitious middle-class people can exploit it by ‘gaming the system’. Violin lessons – check, work placement at uncle’s bank – check, charity work with the disadvantaged – check,fn3 high GPA – check. By contrast, if you hire that brilliant backgammon player, you do know one thing. He’s genuinely talented at something – and it’s unlikely that his parents have spent a fortune on private backgammon lessons. Real excellence can come in odd packaging. Nassim Nicholas Taleb applies this rule to choosing a doctor: you don’t want the smooth, silver-haired patrician who looks straight out of central casting – you want his slightly overweight, less patrician but equally senior colleague in the ill-fitting suit. The former has become successful partly as a result of his appearance, the latter despite it.

Business people and politicians do not quite understand this and tend to evaluate decisions by the rigour of the process that produces them, rather than by the rigour with which you evaluate their consequences. To them, the use of reason ‘looks scientific’, even if it is being used in the wrong place. After all, should we refuse to use antibiotics, X-rays, microwave ovens or pacemakers because the scientific discoveries which led to their creation were the product of lucky accidents?

One astonishing possible explanation for the function of reason only emerged about ten years ago: the argumentative hypothesisfn5 suggests reason arose in the human brain not to inform our actions and beliefs, but to explain and defend them to others. In other words, it is an adaptation necessitated by our being a highly social species. We may use reason to detect lying in others, to resolve disputes, to attempt to influence other people or to explain our actions in retrospect, but it seems not to play the decisive role in individual decision-making. In my view, this theory has much to commend it. For one thing, it explains why individuals use reason so sparingly, selectively and above all self-servingly. It explains why we are good at contriving reasons for positions we already hold, or for decisions we have already made. And it explains confirmation bias, which leads people to seek out and absorb only that information which supports an existing belief. It also explains ‘adaptive preference formation’, where we change our perception of reality in order to depict ourselves in a better light. In this model, reason is not as Descartes thought, the brain’s science and research and development function – it is the brain’s legal and PR department. Understanding this theory seems important, first of all because it might help us see what human reason can and can’t do well.fn6 It might also help us understand how the misuse or overuse of reason can backfire. Collective, self-servi...

I rang a company’s call centre the other day, and the experience was exemplary: helpful, knowledgeable and charming. The firm was a client of ours, so I asked them what they did to make their telephone operators so good. The response was unexpected: ‘To be perfectly honest, we probably overpay them.’ The call centre was 20 miles from a large city; local staff, rather than travelling for an hour each day to find reasonably paid work, stayed for decades and became highly proficient. Training and recruitment costs were negligible, and customer satisfaction was astoundingly high. The staff weren’t regarded as a ‘cost’ – they were a significant reason for the company’s success. However, modern capitalism dictates that it will only be a matter of time before some beady-eyed consultants pitch up at a board meeting with a PowerPoint presentation entitled ‘Rightsizing Customer Service Costs Through Offshoring and Resource Management’, or something similar. Within months, either the entire operation will be moved abroad, or the once-happy call centre staff will be forced on to zero-hour contracts. Soon nobody will phone to place orders because they won’t be able to understand a word they are saying, but that won’t matter when the company presents its quarterly earnings to analysts and one chart contains the bullet point: ‘Labour cost reduction through call centre relocation/downsizing’.

A great deal of the effectiveness of advertising derives from its power to direct attention to favourable aspects of an experience, in order to change the experience for the better. Strangely, there is one form of enhancement to a menu that seems to be the kiss of death: adding photographs of dishes to a menu seems to heavily limit what you can charge for them. Opinion is divided on why. Some people think that the practice is strongly associated with downmarket restaurants, while others believe that attractive photographs may raise expectations too high, leading to inevitable disappointment when the real food arrives.

A knowledge of the human physique is considered essential in designing a chair, but a knowledge of human psychology is rarely considered useful, never mind a requirement, when someone is asked to design a pension scheme, a portable music player or a railway. Who is the Herman Miller of pensions, or the Steve Jobs of tax-return design? These people are starting to emerge – but it has been a painfully long wait. If there is a mystery at the heart of this book, it is why psychology has been so peculiarly uninfluential in business and in policy-making when, whether done well or badly, it makes a spectacular difference.

Although you may think that people instinctively want to make the best possible decision, there is a stronger force that animates business decision-making: the desire not to get blamed or fired. The best insurance against blame is to use conventional logic in every decision. ‘No one ever got fired for buying IBM’ was never the company’s official slogan – but when it gained currency among corporate buyers of IT systems, it became what several commentators have called ‘the most valuable marketing mantra in existence’. The strongest marketing approach in a business-to-business context comes not from explaining that your product is good, but from sowing fear, uncertainty and doubt (now commonly abbreviated as FUD) around the available alternatives. The desire to make good decisions and the urge not to get fired or blamed may at first seem to be similar motivations, but they are, in fact, never quite the same thing, and may sometimes be diametrically different.

it has long surprised observers that, if a customer has a problem and a brand resolves it in a satisfactory manner, the customer becomes a more loyal customer than if the fault had not occurred in the first place. Odd, until you realise that solving a problem for a customer at your own expense is a good way of signalling your commitment to a future relationship. The theory of ‘continuation probability’ would also predict that, when a business focuses narrowly on short-term profit maximisation, it will appear less trustworthy to its customers, something that seems all too plausible.

My contention is that our perception is calibrated more widely in this way. We notice and attach significance and meaning to those things that deviate from narrow, economic common sense, precisely because they deviate from it. The result of this is that the pursuit of narrow economic rationalism will produce a world rich in goods, but deficient in meaning. In architecture this has produced modernism, a style with a marked absence of decoration or ‘spurious’ detail, and a corresponding loss of ‘meaning’.fn2 My secret hope is that, with the 3D-printing of buildings becoming possible, a certain Gaudí-ness may reappear in twenty-first-century buildings.fn3

Quite simply, all powerful messages must contain an element of absurdity, illogicality, costliness, disproportion, inefficiency, scarcity, difficulty or extravagance – because rational behaviour and talk, for all their strengths, convey no meaning.

One of the most important ideas in this book is that it is only by deviating from a narrow, short-term self-interest that we can generate anything more than cheap talk. It is therefore impossible to generate trust, affection, respect, reputation, status, loyalty, generosity or sexual opportunity by simply pursuing the dictates of rational economic theory. If rationality were valuable in evolutionary terms, accountants would be sexy. Male strippers dress as firemen, not accountants; bravery is sexy, but rationality isn’t. Can this theory be extended further? For instance, is poetry more moving than prose because it is more difficult to write?fn3 And is music more emotionally potent than normal speech because it is more difficult to sing than to talk?

Relatively small businesses that might not be able to afford to advertise in any conventional sense, could transform their fortunes by paying a little attention to the workings of psycho-logic. The trick involves simply understanding the wider behavioural system within which they operate. Cafés could boost sales by improving their menu design. Many small shops are inadequately lit, and so passers-by assume they are closed – how much business do they lose as a result?fn4 Pubs are often needlessly intimidating because their windows are made of frosted glass, preventing people from looking inside before entering. Pizza delivery firms could differentiate themselves in a crowded market by agreeing to deliver tea, coffee, milk and toilet paper alongside a pizza. Restaurants might increase sales by allowing the kerbside collection of take-away meals – or by adding a sign which says ‘parking at rear’.

Modern environmentalists also suggest that status-signalling competition between humans is destroying the planet. They propose that the Earth has enough resources to comfortably support the present population if we are all prepared to live modestly, but that natural rivalry can lead to ever-rising expectations – and with it increasing consumption. In many ways, this competition is not healthy – and nor does it necessarily contribute much to human happiness. In some ways it places people under an obligation to spend more money than they would otherwise choose to, just to maintain their status relative to other people.

The psychologist Nicholas Humphrey argues that placebos work by prompting the body to invest more resources in its recovery.fn2 He believes that evolution has calibrated our immune system to suit a harsher environment than the current one, so we need to convince our unconscious that the conditions for recovery are especially favourable in order for our immune system to work at full tilt. The assistance of doctors (whether witch or NHS), exotic potions (whether homeopathic or antibiotic) or the caring presence of relatives and friends can all create this illusion, yet policymakers hate the idea of any solution that involves such unconscious processes – too little is spent on researching the placebo effect in proportion to its importance.

The placebo effect, like many other forms of alchemy, is an attempt to influence the mind or body’s automatic processes. Our unconscious, specifically our ‘adaptive unconscious’ as psychologist Timothy Wilson calls it in Strangers to Ourselves (2002), does not notice or process information in the same way we do consciously, and does not speak the same language that our consciousness does, but it holds the reins when it comes to much of our behaviour. This means that we often cannot alter subconscious processes through a direct logical act of will – we instead have to tinker with those things we can control to influence those things we can’t or manipulate our environment to create conditions conducive to an emotional state which we cannot will into being.

It is this oblique hacking of unconscious emotional and physiological mechanisms that often causes suspicion of the placebo effect, and of related forms of alchemy. Essentially we like to imagine we have more free will than we really do, which means we favour direct interventions that preserve our inner delusion of personal autonomy, over oblique interventions that seem less logical.

Our conscious mind tries hard to preserve the illusion that it deliberately chose every action you have ever taken; in reality, in many of these decisions it was a bystander at best, and much of the time it did not even notice the decision being made. Despite this, it will still construct a story in which it was the decisive actor. For instance, ‘I saw the bus coming and jumped back on the kerb,’ while in fact, you may well have started jumping before you were even consciously aware of the bus. In the words of Jonathan Haidt,fn1 1 ‘The conscious mind thinks it’s the Oval Office, when in reality it’s the press office.’ By this he means that we believe we are issuing executive orders, while most of the time we are actually engaged in hastily constructing plausible post-rationalisations to explain decisions taken somewhere else, for reasons we do not understand. But the fact that we can deploy reason to explain our actions post-hoc does not mean that it was reason that decided on that action in the first place, or indeed that the use of reason can help obtain it.

Someone who has not mastered the technique of oblique influence can only envisage direct intervention to achieve the desired effect, as follows: If you want to change gear, you must move the gearstick. If you want people to work harder, you must pay them more. If you want people to give up smoking, you must tell them it kills them. If you want people to take out pensions, you must give them a tax incentive. If you want people to pay more for your product, you must make it objectively better. If you want to improve a train journey, you must make the trains faster. If you want to improve your well-being, you must consume more resources. If you want people to get better, you must give them an active drug.

To recalibrate our immune response to levels appropriate to the more benign conditions we experience in everyday modern life, it may be necessary to deploy some benign bullshit.fn3 This, I suppose, was what my grandfather was doing in the days before antibiotics, when he cheered up his patients with banter and encouraged them to wrap up warmly, stay in bed, feed themselves well and drink medicinal whiskies – perhaps prescribing for good measure some ineffectual pills that nonetheless created enough of an illusion of optimism for the patient’s body to enter ‘healing mode’.

The only way you can discover what people really want (their ‘revealed preferences’, in economic parlance) is through seeing what they actually pay for under a variety of different conditions, in a variety of contexts. This requires trial and error – which requires competitive markets and marketing.

To those five giant industries that exist by selling mood-altering substances – alcohol, coffee, tea, tobacco and entertainment – should we add the placebo industry? After all, it’s not just the purchase of cosmetics that can be explained in this way – I would contend that a large proportion of consumerism is designed to achieve the same thing. In fact, much luxury goods expenditure can only be explained in this way – either people are seeking to impress each other, or they are seeking to impress themselves.fn1 Is almost everything a mood-altering substance?

If you look at behaviours to hack the unconscious, they all seem to have an element that is wasteful, unpleasant or downright silly. Cosmetics are insanely overpriced and time-consuming to apply. Alcohol, when you think about it, doesn’t really taste very nice: on a really hot day when you are parched, which would you honestly prefer – a glass of Château d’Yquem or a raspberry Slush Puppie? Placebo treatments like homeopathy involve a large amount of ritual and nonsense. Medicine tastes horrid. At some point, we have to ask a vital question: do these various things work despite the fact that they are illogical, or do they work precisely because they are? And if our unconscious instincts are programmed to respond to and to generate behaviours precisely because they deviate from economic optimality, what might be the evolutionary reason for this? It seems rather like the lesson that is taught to aspiring journalists: ‘Dog Bites Man’ is not news, but ‘Man Bites Dog’ is. Meaning is disproportionately conveyed by things that are unexpected or illogical, while narrowly logical things convey no information at all.

Now, I accept that the need to solve ‘narrow context’ problems is much greater today than it was a million years ago, and there’s no denying the contribution that rational approaches have made to our lives – in fields such as engineering, physics and chemistry. But I would also contend that our environment has not changed all that much: most big human problems, and the majority of business decisions, are still ‘wide context’ problems. The problems occur when people try to solve ‘wide’ problems using ‘narrow’ thinking. Keynes once said, ‘It is better to be vaguely right than precisely wrong’, and evolution seems to be on his side. The risk with the growing use of cheap computational power is that it encourages us to take a simple, mathematically expressible part of a complicated question, solve it to a high degree of mathematical precision, and assume we have solved the whole problem. So my GPS answers a narrow question like ‘How long will it take to drive to Gatwick?’ brilliantly, but the wider questions like ‘How should I get there and when should I set off?’ still remain. The GPS device has provided a brilliant answer to the wrong question. In the same way, companies can, for instance, optimise their digital advertising spend to a huge degree of precision, but this does not necessarily mean that they have answered the wider marketing questions, such as ‘Why should people trust me enough to buy what I sell?’

We fetishise precise numerical answers because they make us look scientific – and we crave the illusion of certainty. But the real genius of humanity lies in being vaguely right – the reason that we do not follow the assumptions of economists about what is rational behaviour is not necessarily because we are stupid. It may be because part of our brain has evolved to ignore the map, or to replace the initial question with another one – not so much to find a right answer as to avoid a disastrously wrong one. The unconscious question is not the one we are supposed to ask, and the one that might be ignored when we try to construct rational rules for decision-making.

“decision makers can satisfice either by finding optimum solutions for a simplified world, or by finding satisfactory solutions for a more realistic world. Neither approach, in general, dominates the other, and both have continued to co-exist in the world of management science.”’ Since then, I aver, the balance has shifted. The former approach – creating a simplified model of the world and applying a logical approach – is in danger of overpowering the other, more nuanced approach, sometimes with potentially dangerous consequences: the 2008 financial crisis arose after people placed unquestioning faith in mathematically neat models of an artificially simple reality.

In any complex system, an overemphasis on the importance of some metrics will lead to weaknesses developing in other overlooked ones. I prefer Simon’s second type of satisficing; it’s surely better to find satisfactory solutions for a realistic world, than perfect solutions for an unrealistic one. It is all too easy, however, to portray satisficing as ‘irrational’. But just because it’s irrational, it doesn’t mean it isn’t right.

The primary reason why we have evolved to satisfice in our particularly human way is because we are making decisions in a world of uncertainty, and the rules for making decisions in such times are completely different from those when you have complete and perfect information. If you need to calculate the hypotenuse of a triangle and you know one interior angle and the length of the other two sides, you can be perfectly correct, and many problems in mathematics, engineering, physics and chemistry can achieve this level of certainty. However, this is not appropriate to most of the decisions we have to make. Such questions as whom to marry, where to live, where to work, whether to buy a Toyota or a Jaguar or what to wear to a conference don’t submit to any mathematical solution. There are too many future unknowns and too many variables, many of which are either not mathematically expressible or measurable. Another good example of a decision that has to be taken subjectively is whether we choose to buy an economical or a high-performance car. In general there is a trade-off between these two attributes. Do you sacrifice economy for performance or performance for economy?

Remember, making decisions under uncertainty is like travelling to Gatwick Airport: you have to consider two things – not only the expected average outcome, but also the worst-case scenario. It is no good judging things on their average expectation without considering the possible level of variance.