How Innovation Works: And Why It Flourishes in Freedom

by Matt Ridley

Amara’s Law states that people tend to overestimate the impact of a new technology in the short run, but to underestimate it in the long run.

And the really interesting thing is that cities need fewer petrol stations and miles of electrical cable or road – per head of population – as they get bigger, but have disproportionately more educational institutions, more patents and higher wages – per head of population – as they get bigger. That is to say, the infrastructure scales at a sublinear rate, but the socio-economic products of a city scale at a superlinear rate. And this pattern holds throughout the world wherever Geoffrey West and his colleagues look.

David Warsh, in a book on the history of economics entitled Knowledge and the Wealth of Nations pointed out that Adam Smith himself created a contradiction that he never resolved and that in some form persists to this day. The famous ‘Invisible Hand’ is about the gradual emergence of equilibria in markets, so that neither the producer nor the consumer can improve upon the deal they have got. This implies diminishing returns: as the world settles upon the right price of a widget, so there are no gains to be made. By contrast, Smith’s other idea, the division of labour, implies the opposite: increasing returns. To use his own example, in a pin factory, as workers share out the tasks and get more specialized and innovative in their work, and therefore collectively more productive, so the cost of making pins comes down and down. Both producers and consumers get more for less. The first metaphor therefore implies negative feedback, the second positive. They cannot both be right.

Indeed, the internet only took off when it eventually escaped the clutches of the Defense Department and was embraced by universities and businesses.

Mazzucato also cites the Small Business Innovation Research programme started by Ronald Reagan as an example of government-funded innovation in the private sector. Yet Mingardi points out that this is the very opposite of directed innovation. The programme simply requires all government agencies with an R&D budget over $100m to spend 2.8 per cent of their budget to promote innovation by small- and medium-sized businesses.

By contrast the Soviet Union was a very clear case of an entrepreneurial state, funding a great deal of research centrally, allowing virtually no private enterprise, and the result was a dismal lack of innovation in transport, food, health or any consumer sector, but lots of advances in military hardware.

There is nothing quite so lacking in innovation as the practices and premises of government.

While this can sometimes happen, it is just as often the case that invention is the parent of science: techniques and processes are developed that work, but the understanding of them comes later. Steam engines led to the understanding of thermodynamics, not the other way round. Powered flight preceded almost all aerodynamics. Animal and plant breeding preceded genetics. Pigeon fancying laid the groundwork for Darwin’s understanding of natural selection. Metalworking helped give birth to chemistry. None of the pioneers of vaccination had the foggiest idea how or why it worked. Understanding of the mode of action of antibiotics came long after they were in practical use.

Likewise, in the twenty-first century, as I have documented in Chapter 4, the work that led to the invention of CRISPR gene editing was driven partly by a desire to solve practical problems in the yogurt industry.

For most people today in Western countries, much of the inequality that exists – though not all – is about luxuries, rather than necessities; at least, this is more true than it was in the past, when poor people often starved to death or died of cold and lacked access to simple things like light.

Innovation often comes from outsiders. This is true of individuals as well as organizations. John Harrison was just a Yorkshire clockmaker, and when he solved the problem of establishing longitude by building robust chronometers for ships, the establishment refused to take him seriously for a long time, because he was not a scientific grandee and his solution did not involve advanced astronomy.

Big companies are bad at innovating, because they are too bureaucratic, have too big a vested interest in the status quo and stop paying attention to the interests, actual and potential, of their customers.

Thus the software world is increasingly a place of open, free sharing of innovations, an unfenced prairie. Far from deterring innovation, the effect seems to have been to encourage it. The Linux Foundation now hosts thousands of open-source projects to ‘harness the power of open source development to fuel innovation at unmatched speed and scale’.

But miniaturization did not prove so easy in microfluidics as in semi-conducting. Whereas a transistor became more reliable as it shrank in size, a blood-diagnostic test became less so.

Amazon is a good example of failure on the road to success, as Jeff Bezos has often proudly insisted. ‘Our success at Amazon is a function of how many experiments we do per year, per month, per week. Being wrong might hurt you a bit, but being slow will kill you,’ Bezos once said: ‘If you can increase the number of experiments you try from a hundred to a thousand, you dramatically increase the number of innovations you produce.’

Lockheed Martin pioneered this idea of a high-risk company within a company, licensed to try crazy things in case some of them led to immense rewards. It opened its secret Advanced Development Programs, better known as the ‘skunk works’ in 1943 and produced some of the first jet fighters and high-altitude spy planes.

Desperate to kill a potential good news story coming out of biotechnology, Greenpeace continued to lobby hard against the crop even as it was proved by relentless experiments to be safe and effective. It was in response to this shocking campaign that 134 Nobel Prize-winners called on Greenpeace in 2017 to ‘cease and desist in its campaign against Golden Rice specifically, and crops and foods improved through biotechnology in general’ (150 have now signed the letter). Their call was in vain.

The herbicide glyphosate, also known as Roundup, has come to be a cheap and ubiquitous method of weed control since its invention by a scientist at Monsanto, John Franz, in 1970. It has huge advantages over other weed-killers. Because it inhibits an enzyme found only in plants it is virtually harmless at normal doses to animals, including people, and because it decomposes rapidly it does not persist in the environment. It is far safer than the stuff it replaced, paraquat, which was sometimes used by suicides. Glyphosate has transformed agriculture by allowing farmers to control weeds chemically, rather than by the more ecologically harmful activity of ploughing: it has led to the no-till revolution. This is especially true where crops genetically modified to be glyphosate-resistant are grown.

In the mid-2010s, Thiel made the following observation: ‘I would say that we lived in a world in which bits were unregulated and atoms were regulated.’ Software was evolving through ‘permissionless innovation’, while physical technology was tied down in regulation that largely stifled change.

By 2050, I am convinced, we could have stemmed the rise of allergies and autoimmune disease, largely by recognizing that the cause lies in the lack of parasites, and the lack of diversity of microflora in our guts, to the presence and resistance of which our immune system is adapted. With transplants of microbiota, or supplements of substances that were once supplied by worms and bacteria, we could have banished many autoimmune diseases, perhaps even including autism or other mental conditions. We will have almost certainly banished the problem of antimicrobial resistance with new strategies for keeping one step ahead of lethal bacteria.

The contrast is all the more striking when you look at the science fiction of the 1950s and 1960s, in which transport technology looms large, whereas computers hardly figure at all. The future, we were told, would include routine space travel, supersonic airliners and personal gyrocopters. No mention of the internet, social media or watching movies on mobiles. I recently dug out an old cartoon strip from 1958 about the future called ‘Closer than we think’. One image shows a ‘rocket mailman’, propelled through the air by a personal jetpack, delivering letters to a house. My grandparents had the opposite experience from what my generation has seen: big changes in transport and few in communication.