How Innovation Works: And Why It Flourishes in Freedom

by Matt Ridley

In the ten years from 2008, America’s economy grew by 15 per cent but its energy use fell by 2 per cent. This is not because the American economy is generating fewer products: it’s producing more. It is not because there is more recycling – though there is. It’s because of economies and efficiencies created by innovation.

Ideas are like rabbits. You get a couple and learn how to handle them, and pretty soon you have a dozen. JOHN STEINBECK

Knowledge is both a public good and a temporarily private one. Knowledge is expensive to produce, but can sometimes pay for itself.

The Japanese miracle was a function of private corporations backed by a vast ecosystem of small enterprises. By contrast the Soviet Union was a very clear case of an entrepreneurial state, funding a great deal of research centrally, allowing virtually no private enterprise, and the result was a dismal lack of innovation in transport, food, health or any consumer sector, but lots of advances in military hardware.

There is another problem with the thesis that government is the source of innovation. The argument nearly always turns on the things that governments supposedly invent and then spin out into the private sector. But, if this were the case, would not governments apply them first within government itself? There is nothing quite so lacking in innovation as the practices and premises of government.

Yet there is no doubt that in recent years there has been a growing tendency among politicians to adopt the notion that science is the mother of invention, and that this is the main justification for funding science. This seems to me a pity, not just because it misreads history, but because it devalues science.

Innovation cannot be forced upon unwilling consumers

The fear that innovation destroys jobs has a long history, dating back to General Ludd and Captain Swing in the early 1800s. In 1812 the Luddites went about smashing stocking frames in protest at the introduction of new machinery into the textile industry, taking their inspiration and their name from a probably apocryphal story of one Ned Ludd who had supposedly done the same back in 1779. In 1830, in a protest at conditions in the farming industry, rioting labourers began burning hay ricks and smashing threshing machines, under a mythical leader named Captain Swing.

In short, the idea that innovation destroys jobs comes around in every generation. So far it has proved wrong. Over the past two centuries productivity in agriculture dramatically increased, but farm workers moved to cities and got jobs in manufacturing. Then productivity in manufacturing rocketed upwards, freeing huge numbers of people to work in services, yet still there was no sign of mass unemployment. Candles were replaced by electric lights, but wick trimmers found other work. Millions of women joined the paid workforce, at least partly as a result of innovation in washing machines and vacuum cleaners, which freed them from much household drudgery, yet employment rates went up not down.

One especially influential study, by Carl Frey and Michael Osborne, published in 2013, came to the conclusion that 47 per cent of all jobs in America are at ‘high risk’ of automation within a ‘decade or two’. However, the OECD re-examined the issue, used a more appropriate database and concluded that a much less frightening 9 per cent of jobs were at risk of disappearing because of automation, and even these would be accompanied by expanding employment in other occupations.

From Thomas Newcomen to Steve Jobs, again and again the great innovators have come from obscure origins, in unfashionable provinces and without good contacts or education.

Big companies are bad at innovating, because they are too bureaucratic, have too big a vested interest in the status quo and stop paying attention to the interests, actual and potential, of their customers.

The one thing that does make a big company innovate is competition.

The ultimate open-source innovation is that done by consumers themselves. Eric von Hippel of Massachusetts Institute of Technology argues that free innovation by the consumer is a neglected sector of the economy and the assumption that innovation is driven by producer innovation is misleading. He calculates that tens of millions of consumers spend tens of billions of dollars every year developing or modifying products for their own use. Most do so during their free time and share them freely with others.

The opportunities for free innovation are growing as computerized design tools and lowered communication costs allow people to do at home what once required a corporate laboratory to achieve.

Free innovators rarely seek patents or copyrights, which means they are willing to share ideas freely.

We need big failures in order to move the needle. If we don’t, we’re not swinging enough. You really should be swinging hard, and you will fail, but that’s okay. JEFF BEZOS

A modern magnetic-levitation train such as Japan’s Chuo Shinkansen uses more, not less, energy than one on rails. Putting one in a vacuum saves energy, but brings a penalty too, because not only do the vacuum pumps need power, but since air resistance cannot help to slow the train, braking requires more energy.

Google, likewise, tolerates and even encourages failure.

When a new invention is first propounded in the beginning every man objects and the poor inventor runs the gauntloop of all petulant wits. WILLIAM PETTY, 1662

In 1655 an apothecary named Arthur Tillyard founded the Oxford Coffee Club for students to discuss ideas over hot drinks at what might be called his ‘café’. Seven years later the club became the Royal Society, Britain’s science academy.

Coffee reached Constantinople in the 1550s and was promptly banned by Sultan Selim II. It was banned again in 1580 by the usurper Murad III and again in the 1630s by Murad IV. This implies that the bans failed each time, but why were these rulers so keen to stamp out this drink? Mainly because coffee houses were places of gossip and therefore potential sedition. Murad III was paranoid that the fact that he had killed his whole family to claim the throne might have been a topic of conversation in coffee houses. I dare say he was right and the subject did sometimes crop up. In 1673 King Charles II of Scotland and England tried to ban coffee houses and was splendidly honest about why he was such a keen prohibitionist: As for coffee, tea and chocolate, I know no good they do; only the places where they are sold are convenient for persons to meet in, sit half day and discourse with all companies that come in of State matters, talking of news and broaching of lies, arraigning the judgments and discretion of their governors, censuring all their actions, and insinuating into the ears of the people a prejudice against them; extolling and magnifying their own parts, knowledge and wisdom, and decrying that of their rulers; which if suffered too long, may prove pernicious and destructive.

Here we see all the characteristic features of opposition to innovation: an appeal to safety; a degree of self-interest among vested interests; and a paranoia among the powerful. Recent debates about genetically modified food, or social media, echo these old coffee wars.

Innovation and Its Enemies, Calestous Juma

Juma chronicles how hansom cab operators in London furiously denounced the introduction of the umbrella; how obstetricians long rejected the use of anaesthesia during childbirth; how musicians’ unions temporarily prevented the playing of recorded music on the radio; how the Horse Association of America for many years fought a rearguard action against the tractor; how the natural-ice harvesting industry frightened people with scares about the safety of refrigerators. Truly, there is likely to be a backlash against any new technology, usually driven partly by vested interests but clothing itself in the precautionary principle.

Since 2005 Canada has approved seventy different transgenic varieties of crops while the European Union has approved just one, and that took thirteen years, by which time the crop was outdated.

Greenpeace turned its attention to blocking the humanitarian application of GMOs, especially in the form of beta-carotene-containing Golden Rice, specially designed by a non-profit project to prevent malnutrition and death among poor children. Golden Rice was developed by the Swiss-based scientist Ingo Potrykus and colleagues, in a long and laborious endeavour during the 1990s, purely as a humanitarian, non-profit project designed to alleviate the high mortality and morbidity caused by vitamin-A deficiency in people who rely on rice for food. By one estimate vitamin-A deficiency kills 2,000 mainly urban children under five every day, 700,000 a year, in countries where poor people eat rice as a staple diet: it lowers their immune resistance and causes them to go blind.

134 Nobel Prize-winners called on Greenpeace in 2017 to ‘cease and desist in its campaign against Golden Rice specifically, and crops and foods improved through biotechnology in general’ (150 have now signed the letter).

Demonization of biotechnology led to a vicious circle as far as the companies involved were concerned. The more the activists demanded regulation and caution, the more expensive it became to develop new crops, and the more therefore it became impossible to do so except within large companies. There was thus a strange symbiosis between big industry and its critics. At one point activists demanded that Monsanto make crops that could not survive beyond the first generation, so they could not run riot in the wild as ‘superweeds’ – an entirely false fear anyway since most crops are ill suited to being weeds. In response, Monsanto therefore explored the possibility of developing genetic variants that were incapable of breeding true. Although it did not develop these, activists immediately accused it of introducing ‘terminator technology’ to trap farmers into buying new seed each year. The charge stuck.

Mark Lynas’s verdict on the GMO episode is unsparing: ‘We permanently stirred public hostility to GMO foods throughout pretty much the entire world, and – incredibly – held up the previously unstoppable march of a whole technology. There was only one problem with our stunningly successful worldwide campaign. It wasn’t true.’ As with the opposition to coffee, it is now clear that the opposition to genetically modified crops was wrong both factually and morally. The technology was safe, environmentally beneficial and potentially good for small farmers. The anti-GM movement caught on amongst wealthy people with plentiful, cheap food. It was not pressing and relevant to their lives to increase crop yields. Those who paid the opportunity cost of the prohibition were the sick and starving who had no voice.

In 2015, however, a World Health Organization body called the International Agency for Research on Cancer (IARC) came to the conclusion that glyphosate might be capable of causing cancer at very high doses. It admitted that by the same criteria, sausages and sawdust should also be classified as carcinogens, while coffee was even more dangerous (and unlike glyphosate is regularly drunk). The effect would be minuscule: Ben & Jerry’s ice cream was found to contain glyphosate at a concentration of up to 1.23 parts per billion, so a child would have to eat 3 tonnes of it a day before any risk would be encountered.

In July 1945 J. K. Jett, the head of the US Federal Communications Commission, gave an interview to the Saturday Evening Post in which he said that millions of citizens would soon be using ‘hand-held talkies’.

However, in 1947 the very same FCC rejected AT&T’s application to start a cellular service, arguing that it would be a luxury service for the few. Television was the priority and got the lion’s share of the spectrum.

In short, the government, in cahoots with crony-capitalist firms with huge vested interests, made the development of cellular service impossible for almost four decades. Who knows what improvements in technology and changes in society it thus prevented?

Thus the story of mobile phones is one in which governments resisted an innovation, in league with vested interests in the private sector. We use smartphones today not because of government regulators, but despite them.

The justification for intellectual property – patents and copyright – is that it is necessary to encourage investment and innovation. With real property rights, people will not usually build a house unless they own the land on which it stands; so they will not invent a drug or write a book unless they can own it.

Most people create works of art because they seek influence or fame, as much as money. Shakespeare had no copyright protection, and pirated copies of his plays abounded, but he still wrote. Today, wherever intellectual-property protections are absent or leaky – in the music industry, for example, where ‘piracy’ has prevailed – there is no diminution in the enthusiasm of creators.

In 2011 the economist Alex Tabarrok argued in his book Launching the Innovation Renaissance that the American patent system, far from encouraging innovation, is now discouraging it. Echoing the famous Laffer curve, which shows that beyond a certain point higher tax rates generate less revenue, he drew a graph on a paper napkin to suggest that beyond a certain point stronger patents generate less innovation, because they make it hard to share ideas, and create barriers to entry. The Semiconductor Chip Protection Act of 1984 resulted in more patenting but less innovation in the United States, as semiconductor firms effectively set about stocking ‘war chests’ of patents to deploy in disputes with each other.

Finally, patents tend to favour inventions rather than innovations: upstream discoveries of principles, rather than downstream adaptation of devices to the market. This has led to the proliferation of what are known as patent thickets: vague intellectual property hedges that block the progress of people trying to move through the intellectual landscape and develop new products. This is a special problem in biotechnology, where innovators often find themselves infringing patents taken out by others relating to molecules they need to use in just a small part of their work. Startups find themselves blocked from following a lead into a new molecular pathway by the unpleasant discovery that another firm has already vaguely patented the use of one of the molecules therein.

The resistance to Uber from taxi drivers, say Lindsey and Teles, is a ‘powerfully vivid illustration of the conflict between occupational licensing and innovation’. Cities such as Paris and Brussels passed laws to restrict or even ban Uber.

Why did the European Commission depart from international practice? The answer emerged via documents revealed under Freedom of Information. The big German manufacturers of vacuum cleaners with bags had been busily lobbying the European Commission. Bagged vacuum cleaners have to increase power usage as they become clogged with dust or they will perform worse. It is a classic case of crony-capitalism: a company lobbying to get the rules written to favour an incumbent technology over an innovative one.

Regulatory shenanigans cause harm not just by suppressing entrepreneurial energy but also by misdirecting it. The economist William Baumol has argued that if the policy background means that the best way to get rich is by building a new device and selling it, then entrepreneurial energy will flow into innovation, but if it is simpler to profit from lobbying government to set the rules up in favour of an existing technology, then all the entrepreneurial energy will go into lobbying.

Innovation is one of those things that everybody favours in general, and everybody finds a reason to be against in particular cases. Far from being welcomed and encouraged, innovators have to struggle against the vested interests of incumbents, the cautious conservatism of human psychology, the profitability of protest, and the barriers to entry erected by patents, regulations, standards and licences.

We wanted flying cars; instead we got 140 characters. PETER THIEL

The main ingredient in the secret sauce that leads to innovation is freedom. Freedom to exchange, experiment, imagine, invest and fail; freedom from expropriation or restriction by chiefs, priests and thieves; freedom on the part of consumers to reward the innovations they like and reject the ones they do not. Liberals have argued since at least the eighteenth century that freedom leads to prosperity, but I would argue that they have never persuasively found the mechanism, the drive chain, by which one causes the other. Innovation, the infinite improbability drive, is that drive chain, that missing link.

This reliance on freedom explains why innovation cannot easily be planned, because neither human wishes nor the means of their satisfaction are easy to anticipate in the detail required; why innovation none the less seems inevitable in retrospect, because the link between desire and satisfaction is only then manifest; why innovation is a collective and collaborative business, because one mind knows too little about other minds; why innovation is organic because it must be a response to an authentic and free desire, not what somebody in authority thinks we should want; why nobody really knows how to cause innovation, because no one can make people want something.

If some predictions about the potential of treatments for the ageing process itself are right, based on a growing understanding of how to clear out senescent cells from tissues, then the cost of care for the elderly may plummet. By 2050, too, we could have experienced the long-promised ‘compression of morbidity’ by which people spend a longer time living but a shorter time dying. So far this has eluded us, as we prevent and cure sudden killers like heart disease much faster than we prevent or cure gradual killers like cancer, let alone chronic diseases like dementia. Medical innovation will surely make it possible to live better lives into old age, with senolytic drugs, robotic keyhole surgery, stem-cell treatment and gene-edited cancer treatments, to name but a handful of possibilities.

By 2050 we could have replenished the ecosystems of the oceans and repaired the rain forests through innovative policies as well as innovative machines. Growth increasingly means getting more benefits from fewer resources, as exemplified by the dematerialization of the economy.

All this (and much more) is easily within the reach of innovation by the next generation of entrepreneurs. But will we let them do it, or are we strangling the golden goose of innovation?

Economists such as Tyler Cowan and Robert Gordon have likewise argued that we are no longer inventing things that really change the world, like toilets and cars, but increasingly playing with trivia like social media.