How Innovation Works: And Why It Flourishes in Freedom

by Matt Ridley

innovation is the most important fact about the modern world, but one of the least well understood.

Much innovation preceded the science that underpinned it.

scholars ‘know remarkably little about the kind of institutions that foster and stimulate technological progress’.

But the striking thing about innovation is how mysterious it still is. No economist or social scientist can fully explain why innovation happens, let alone why it happens when and where it does.

It runs mostly on trial and error, the human version of natural selection. And it usually stumbles on great breakthroughs when looking for something else: it is heavily serendipitous.

Innovation often disappoints in its early years, only to exceed expectations once it gets going, a phenomenon I call the Amara hype cycle, after Roy Amara, who first said that we underestimate the impact of innovation in the long run but overestimate it in the short run.

And it was a gradual, stumbling change, with no eureka moment. These features are typical of innovation.

Innovation seems so obvious in retrospect but is impossible to predict at the time.

The editors of a journal called The Chemist had this to say, rather perceptively: ‘He is distinguished from other public benefactors, by never having made, or pretended to make it his object to benefit the public . . . This unpretending man in reality conferred more benefit on the world than all those who for centuries have made it their especial business to look after the public welfare.’

The truth is that twenty-one different people can lay claim to have independently designed or critically improved incandescent light bulbs by the end of the 1870s, mostly independent of each other, and that is not counting those who invented critical technologies that assisted in the manufacture of light bulbs, such as the Sprengel mercury vacuum pump.

he was none the less the first to bring everything together, to combine it with a system of generating and distributing electricity, and thereby to mount the first workable challenge to the incumbent technologies of the oil lamp and the gas lamp.

Edison’s approach worked: within six years he had registered 400 patents. He remained relentlessly focused on finding out what the world needed and then inventing ways of meeting the needs, rather than the other way around. The method of invention was always trial and error.

To single out clever people who made the difference along the way is both difficult and misleading. This was a collaborative effort of many brains. Long after the key technologies had been ‘invented’, innovation continued.

So trial and error was vital to innovation in fracking. Steinsberger made a series of lucky mistakes, failing many times along the way. And when he had found the formula, he did not know why it worked.

In this case science came in behind the technology, rather than vice versa.

innovation proves to be gradual and to begin with the unlettered and ordinary people, before the elite takes the credit.

Vaccination exemplifies a common feature of innovation: that use often precedes understanding.

So the source of the invention of chlorination, like that of vaccination, is enigmatic and confused. Only in retrospect can it be seen as a disruptive and successful innovation that saved millions of lives. It evolved rather slowly, probably from serendipitous beginnings in largely mistaken ideas.

penicillin, or ‘mould juice’ as he initially called it.

The story of penicillin reinforces the lesson that even when a scientific discovery is made, by serendipitous good fortune, it takes a lot of practical work to turn it into a useful innovation.

In short, it was the product of incremental tinkering and trial and error by several people, not of brilliant leaps of imagination by a genius.

The story of the screw propeller shows all the usual elements of an innovation: a long prehistory, simultaneous breakthroughs by two rivals, then incremental evolution over many years.

A collision knocked one turn of the screw off, after which the boat went much faster, an unexpected discovery related to turbulence and drag.

The story of the internal-combustion engine displays the usual features of an innovation: a long and deep prehistory characterized by failure; a shorter period marked by an improvement in affordability characterized by simultaneous patenting and rivalries; and a subsequent story of evolutionary improvement by trial and error.

reminder that in the early years of the car industry, as in the early years of computers, mobile phones and many other innovations, the inventors thought they were developing a luxury good for the upper-middle classes.

innovation in materials is vital to realizing an advance that can be conceived but not built.

Like radar and the computer, the jet is often thought to be a product of wartime inventiveness. But, as in those other cases, the key work was actually done long before hostilities broke out, both in Britain and in Germany, and it is impossible to know just how fast the jet would have been developed and commercialized in an alternative universe in which the 1940s were prosperous and peaceful.

it is the widespread use of dull, low-tech but vital practices such as ‘crew resource management’ techniques, and checklists galore, with cross-checking between crew members, and a culture of challenge, that have made the big difference since the 1970s.

The astonishing safety record of the modern airline industry has been achieved, quite literally, by trial and error. Its methods have since been emulated in other walks of life such as surgery and offshore oil and gas exploration.

the English turned this into a belief that potatoes were Roman Catholic agents: in Lewes, in Sussex, the crowd shouted: ‘No potatoes, no popery!’ during an election in 1765.

the result of Frederick the Great’s wars was that the potato, unknown or despised in most of central and eastern Europe in 1700, had by 1800 become an indispensable part of the European diet.

He planted a field of potatoes on the outskirts of Paris and posted guards to protect it, knowing that the presence of the guards would itself advertise the value of the crop, and attract hungry thieves at night, when the guards were mysteriously absent.

Desperate to get access to guano, the American Congress passed an Act saying that any American who found a guano island in the Pacific could claim it for the United States – which is why so many mid-Pacific atolls belong to America today.

As Hager recounts, none of these challenges could be overcome without access to the ideas being developed in other industries, a fine example of how innovation thrives in an ecosystem of innovation:

The Indian bureaucrats were adamant that Mexican wheats should not even be allowed in the country, let alone encouraged. The biologists warned of devastation and disease if the wheats failed. The social scientists warned of ‘irreversible social tensions’ and riots if the wheats succeeded – and caused some farmers to make more money than others. Thus do innovation’s opponents seek any argument, however absurd, to defend the status quo.

This fifty-year story of how dwarfing genes were first found in Japan, cross-bred in Washington, adapted in Mexico and then introduced against fierce opposition in India and Pakistan is one of the most miraculous in the history of humankind.

Between 1960 and 2010, the acreage of land needed to produce a given quantity of food has declined by about 65 per cent.

By the middle of the current century, the world will be feeding nine billion people from a smaller area of land than it fed three billion from in 1950.

When zero is added to a number or subtracted from a number, the number remains unchanged; and a number multiplied by zero becomes zero. BRAHMAGUPTA, the year 628

Zero turns numbers from adjectives into nouns, and becomes a number in its own right. This was an innovation of far-reaching consequence, for sure, but it involved no technology.

The name ‘Fibonacci’ was coined in the nineteenth century as a contraction of the phrase ‘filius Bonacci’ or son of a good bloke,