Tim Harford's Blog, page 41

How to vaccinate the world more quickly

With more than half the UK population fully vaccinated and the UK government just a little too eager to declare victory, spare a thought for Cameroon. With a population about half the size of England, Cameroon has — according to Our World in Data — administered just 160,000 doses of vaccine. On a good day, the UK manages that many before lunch.

I have a certain romantic attachment to Cameroon, but the west African nation is not alone in lacking vaccines. More than six months into the global vaccination campaign, fewer than a quarter of people around the world have received even a single dose of a vaccine. It is no wonder that more people have already died of Covid in 2021 than died from the disease in 2020.

So what can be done? There has been much talk about vaccine equity, but the main problem is not vaccine hoarding or price-gouging. It is that manufacturers can’t make doses fast enough. (If they could, then India, a huge vaccine producer, would have fully vaccinated more than 5 or 6 per cent of its population by now.)

Global production has been impressive and accelerating: according to Airfinity, a life sciences analytics company, the billion-dose production mark was hit only on April 12. A billion more were produced by May 26 and the third billion by June 22. That’s good. But we need 11 billion doses to fully vaccinate 70 per cent of the world, which may not happen until 2022.

Recently, however, some momentum has built up behind an idea that sounds almost childishly simple: if we reduce the dose size, we can vaccinate more people from each vial of vaccine. Why not give people half doses? What about quarter doses? With quarter doses, we could have already vaccinated the world’s adult population.

The idea seems absurd — you can’t get drunk more cheaply by diluting your beer — but it all depends on how effective lower doses might be. Five years ago, faced with a yellow-fever outbreak and a shortage of vaccines in the Democratic Republic of Congo, seven million people received one-fifth of a dose each. The strategy, endorsed by the WHO, seems to have worked.

Alex Tabarrok, a professor at George Mason University, has been pushing the idea of alternative dosing regimes for several months. Recently, he and other researchers, including vaccine market specialist and Nobel laureate economist Michael Kremer, released a working paper exploring the issue.

At the same time, a letter advocating trials of fractional doses, written by epidemiologists Benjamin Cowling and Wey Wen Lim and a virus evolution specialist Sarah Cobey, has been published in Nature Medicine.

What evidence is there low-dose vaccinations might also work for Covid? From full-scale clinical trials, not much — although there was the serendipitous discovery that the Oxford/AstraZeneca vaccine seemed to work better when the first injection consisted of a half dose.  But there is plenty of data on the antibody levels that people produce in response to small doses, and, according to a recent paper in Nature Medicine by David Khoury and colleagues, those antibodies are strongly correlated with real-world protection against Covid. 

As Kremer and his colleagues observe, if antibody levels really are a good measure of protection, then the mRNA vaccines (BioNTech/Pfizer and Moderna) may protect as well as the highly effective AstraZeneca vaccine even if they are deployed at two-thirds, 50 per cent or even 25 per cent strength.

A recent preliminary report, yet to be properly reviewed, also finds that a two-dose course of Moderna’s vaccine at 25 per cent strength produces an antibody response comparable with that of a case of Covid. A trial is under way in Belgium exploring alternative doses of the Pfizer vaccine, while Moderna has said it is also investigating lower doses. 

The concept of a standard or full dose is fuzzier than one might imagine. These vaccines were developed at great speed, with a focus on effectiveness that meant erring towards high doses. Melissa Moore, a chief scientific officer at Moderna, has acknowledged this. It is plausible that we will come to regard the current doses as needlessly high.

This isn’t to say that we should abandon the standard doses, which have been tried and tested in large clinical trials. But it does suggest that we should test alternatives immediately. Is there a downside? If low-dose vaccines don’t work as well as studies of antibodies suggest, that is a problem that booster shots should be able to fix. 

More worrying is the prospect that a large pool of low-dosed people might push the virus to evolve towards vaccine resistance. Cobey acknowledges that risk but argues that if fractional doses help to reduce the number of infected people, that gives the virus fewer opportunities to mutate. Dangerous mutations might be reduced, rather than increased — it is unclear.  What is clear is that millions of lives might be saved if fractional dose vaccines are proved to work well.

There is a lesson to be learnt here. While the production and testing of these vaccines has been little short of miraculous, next time we can do better. We should run more trials, earlier, to produce evidence on a wider variety of questions than “Are they safe?” and “Do they work?” 

Such trials are expensive; the understanding they produce is worth paying for.

Written for and first published in the Financial Times on 14 July 2021.

The paperback of “How To Make The World Add Up” is now out. US title: “The Data Detective”.

“One of the most wonderful collections of stories that I have read in a long time… fascinating.”- Steve Levitt (Freakonomics)

“If you aren’t in love with stats before reading this book, you will be by the time you’re done.”- Caroline Criado Perez (Invisible Women)

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

We must face facts, even the ones we don’t like

The recent unsettling footage of England’s chief medical officer Chris Whitty being grabbed and harassed in a central London park brought to mind many things. There were the similar scenes of BBC journalist Nicholas Watt being pursued, surrounded and abused at a protest in Westminster. Darker still, there was the murder of the MP Jo Cox during the Brexit campaign five years ago.

But I was also reminded of the square root of two.

Two and a half thousand years ago, followers of Pythagoras believed that the constants of the universe were constructed of whole numbers. The Pythagoreans were wrong. One such constant is a simple diagonal across a square — the square root of two. But there are no two whole numbers which, as a fraction, give us the square root of two. 3⁄2 isn’t far off. 10⁄7 is closer. But you can give up on finding the exact fraction.*

None of this should cause much alarm, except perhaps to the long-suffering editors and typesetters of this column. Yet the Pythagoreans were extremely perturbed by the simple demonstration that a basic constant could not be expressed as a fraction of whole numbers. So perturbed, in fact, that it is said the mathematician Hippasus was murdered by being thrown overboard while at sea as a punishment for discovering the ghastly truth.

We don’t know much about Hippasus, and the evidence that he was killed for this little proof is rather patchy. But perhaps the legend has lived on because it serves as a cautionary tale. When a society persecutes people for telling the truth, it is a short step from losing its collective mind.

The totalitarian regimes of the 20th century give us the starkest examples of such insanity. Stalin persecuted genetics researchers in the 1930s and ostentatiously praised the scientist Trofim Lysenko when he claimed that genetics was a “bourgeois perversion” and geneticists were “saboteurs”. The resulting crop failures killed millions. For an encore, Stalin ordered the killing of the statistician in charge of the 1937 census, Olimpiy Kvitkin. Kvitkin’s crime was that his census revealed a fall in population as a result of that famine. Telling that truth could not be forgiven.

In May, the great crop scientist Yuan Longping died at the age of 90. He led the research effort to develop the hybrid rice crops that now feed billions of people. Yet in 1966, he too came very close to being killed as a counter-revolutionary during China’s cultural revolution.

In western democracies we do things differently. Governments do not execute scientists; they sideline them. Late last year, Undark magazine interviewed eight former US government scientists who had left their posts in frustration or protest at the obstacles placed in their way under the presidency of Donald Trump.

Then there are the random acts of hostility on the street and the death threats on social media. I have seen Twitter posts demanding that certain statisticians be silenced or hunted down and destroyed, sometimes for doing no more than publishing graphs of Covid-19 cases and hospitalisations. Even when this remains at the level of ugly intimidation, it is horrible to hear about and must be far worse to experience. It is not something we should expect a civil servant, a vaccine researcher or a journalist to have to endure. And it would be complacent to believe that the threats are always empty.

What can be done? We can demand better from our leaders. Trump never tired of winking his approval at violence against journalists; we can but hope that future presidents refrain. In September 2019, Boris Johnson was dismissive of complaints from MPs who had received death threats. The solution, he said, was Brexit. He has been much quicker to support Whitty, but it seems that his opposition to intimidation and harassment is rather more conditional than one might wish.

While a firmer moral lead from our politicians would help, ultimately the respect for facts — and those who research or report them — has to come from all of us. The facts are sometimes unpleasant: Brexit creates trade barriers between large neighbouring economies. Carbon dioxide emissions are seriously altering the climate. Sars-Cov-2 is much more dangerous than seasonal flu, and cases are rising dramatically in the UK. It would be nice if none of these things were true, but the vast majority of us are adult enough to accept the evidence, the expert judgment of those who gather that evidence and the honesty of those who report it.

The message can be infuriating, but let’s not throw the messenger overboard.

 

* Prove it, you say? Assume a whole-number fraction, a⁄b, does equal √2. Let’s also assume that a⁄b is the simplest possible fraction, with a and b sharing no common factors. Rearranging a⁄b = √2 gives us 2b2 = a2. That means a2 is an even number, which implies four things: a is also even, and therefore a2⁄2 is also even, and therefore b2 is even, and therefore b is even. Alas, we began by assuming that a⁄b was the simplest possible whole-number fraction, but we’ve just proved that a⁄b is the ratio of two even numbers and therefore the fraction could be simplified by dividing both of them by two. This contradiction shows that our original assumption — that a and b exist at all — must be wrong.

Written for and first published in the Financial Times on 9 July 2021.

The paperback of “How To Make The World Add Up” is now out. US title: “The Data Detective”.

“One of the most wonderful collections of stories that I have read in a long time… fascinating.”- Steve Levitt (Freakonomics)

“If you aren’t in love with stats before reading this book, you will be by the time you’re done.”- Caroline Criado Perez (Invisible Women)

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

4000 weeks and the clock is ticking

Oliver Burkeman’s wonderful (and alarming) new book is “4000 Weeks”. I wholeheartedly recommend it, despite the stress-inducing reminder that the human lifespan is 4000 weeks and I’m well past half way.

Burkeman’s book is part Getting Things Done, part Being and Time, and part The Tao of Pooh. He takes seriously the self-help literature on time management (he’s read pretty much every self-help book going so that you don’t have to), but also takes seriously the fact that there will never be a moment when you’ve cleared the decks and ticked off everything on the To Do list. Borrowing from Borges, Burkeman points out that while we experience time flowing past us, we are the passing of time. The river is always flowing and there will never be a moment when we are able to scramble out and sit in serenity on the bank. There’s plenty of philosophy in the book (western, Taoist and Buddhist) but it is extremely well-written and easy to read.

Several ideas stuck with me, but to pick one out, Burkeman warns against the “causal catastrophe” – treating every moment of your life as a means to some future end. You can study to pass your exams to get a job to get an income to get a house, or take up running in order to achieve the goal of running a marathon, or parent your babies and toddlers with the aim in mind of creating happy well-adjusted adults – but it is dangerous and possibly tragic to treat everything as a step on the road to something else. The cliché is that it’s the journey, not the destination – but Burkeman has a way of delivering an intellectual slap in the face that reminds the reader that behind the cliché is an important truth.

The paperback of “The Next 50 Things That Made The Modern Economy” is FINALLY coming – 26 August 2021. Please think about pre-ordering, which is hugely helpful in stimulating bookshops to stock and display the book.

“Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”- Bill Bryson

“Harford is a fine, perceptive writer, and an effortless explainer of tricky concepts. His book teems with good things, and will expand the mind of anyone lucky enough to read it.”- The Daily Mail

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

Statue wars, pandemic reopening, and the art of less

Weniger, aber besser. These three words — less, but better — summarise the philosophy of the great German designer Dieter Rams. His striking designs, from Braun electronics to Vitsoe furniture, have been influential to the point of ubiquity. Apple’s original iPod clearly resembles a Rams-designed radio.

But while “less, but better” is revered by designers, it’s not the way most of us live our lives. Our homes are full of junk, our diaries are full of meetings and our attention is fragmented by dozens — hundreds? — of electronic interruptions a day. Countercultural counter-clutter manifestos have been popular: Greg McKeown’s Essentialism (get rid of unnecessary tasks and meetings), Cal Newport’s Digital Minimalism (get rid of unnecessary apps and devices) and of course Marie Kondo’s The Life-Changing Magic of Tidying (get rid of unnecessary possessions). But like Rams himself, they are swimming against the tsunami of digital, physical and mental “stuff”.

Why do we accumulate so much? An intriguing explanation comes from one of the oldest ideas in behavioural economics: the “endowment effect”. The term was coined by one of the fathers of the field, Nobel laureate Richard Thaler. In his book Misbehaving, Thaler described wine connoisseur Richard Rossett’s cellar, which contained bottles he had purchased for a few dollars that had matured into wines worth hundreds. Rossett occasionally drank these fine vintages, yet he would never add to his cellar by buying wines at high prices, nor would he sell those he already had at a huge profit.

There is an inconsistency here: wine cannot logically be both too expensive to buy and too cheap to sell. This is the endowment effect, by which we value possessions in part because they are possessions. Still, only an economist would find Rossett’s behaviour odd. (Compounding the mystery, Rossett was the head of the economics department at the University of Rochester.)

While Rossett’s case is an intuitive example, Thaler, Jack Knetsch and Daniel Kahneman also produced experimental evidence of the effect. In one study of students, half were given a commemorative mug. All were told to write down the price at which they would be willing to sell their mug — or to buy a mug if they started without one. Those with a mug were reluctant to sell for $5. Those without one were reluctant to buy at half that price. This endowment effect suggests that the status quo matters far more than it should. Often we hold on to things for no reason other than that they are our things.

Minimalists understand the power of the status quo and work to counteract it. Newport, for example, argues that the minimalist should begin with a month-long period of digital fasting: only the most essential tools are to be allowed. Everything else must go. This is not intended as a “detox”. It’s a blank slate, designed to change the status quo. At the end of this period, says Newport, digital tools should be allowed back in only as a deliberate choice, rather than because we sleepwalked into using them once and never let go.

Kondo also fights the status quo. She advocates removing possessions from their usual setting and piling them all together, a bracing experience that reminds us just how much unnecessary stuff most of us own. Then, argues Kondo, look for what “sparks joy”. Deciding to keep something from the pile becomes an active choice rather than a resigned acceptance of the status quo.

I was reminded of this as I pondered the argument over all the portraits and statues associated with the UK’s colonial past. Nobody can pretend to resolve this with a single proposal, but it does strike me that we’d be in an easier place if we occasionally made like Kondo and took them all down. We could put all the portraits and all the statues in a big pile in the centre of each town or the lobby of each grand building. Then we could make an active choice as to who we really wanted on the pedestal for the next quarter of a century. Does Edward Colston really spark joy? Does Cecil Rhodes? Everyone who misses the cut could be stored away until a future round.

I can’t imagine that happening to the portraiture of a Cambridge college or the statues in Trafalgar Square any time soon. Deliberately stepping away from the status quo is not always desirable and it is rarely easy. But most of us have had to do just that over the past 15 months.

Remember diary squeezes? Juggling the school run with an exercise class? The embarrassment of double-booking a dinner with friends and a night at the theatre? Neither do I. But logically these things must once have happened and they’re starting to happen again. In the desperation to get back to normality, to see people (anyone) and go places (anywhere), there’s a risk that we miss the Kondo window of opportunity in which things have been reset and the endowment effect does not exist.

I am trying to think, rather than simply revert to the status quo. Not every task on my To Do list and every meeting in my calendar sparks joy, but I try.

Less, but better.

Written for and first published in the Financial Times on 2 July 2021.

The paperback of “How To Make The World Add Up” is now out. US title: “The Data Detective”.

“One of the most wonderful collections of stories that I have read in a long time… fascinating.”- Steve Levitt (Freakonomics)

“If you aren’t in love with stats before reading this book, you will be by the time you’re done.”- Caroline Criado Perez (Invisible Women)

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

Summer maths books!

New questions have gone live on the Data Detective challenge on the Good Judgement Open website – check them out and test your skills!

I’ve found myself reading four very interesting maths books this summer.

Shape by Jordan Ellenberg – I’m speaking on a panel with Jordan at the San Diego Union-Tribune festival of books and so I thought I should catch up on his book. (He is the author of the excellent How Not To Be Wrong.) Shape argues that geometry is everywhere, and awesome. Ellenberg writes about maths with admirable clarity but he’s also genuinely funny, which is a real plus.

The Maths of Life and Death by Kit Yates. Less pandemic maths than you might fear, don’t worry! A lively tour of all sorts of mathematical ideas from catch-and-release in snails to the millennium bug.

Thinking better: The Art of the Shortcut by Marcus du Sautoy. Marcus frames mathematics as the art of shortcuts – reaching your destination with less time and effort – and ranges across calculus, statistics, geometry. It’s a lovely idea and very well executed. I’m not so sure of the interleaving of the mathematical chapters with short interdisciplinary interviews. For example, having outlined the art of the pattern-recognising shortcut, Marcus (an accomplished musician) interviews the brilliant cellist Natalie Clein. Is there a shortcut to mastering the cello, he wonders? Um – no.

The Art of More by Michael Brooks – I enjoyed this one a lot. Like Du Sautoy, Brooks is arguing that mathematics helps us solve all sorts of important practical problems. What distinguishes Brooks’s approach is that he situates the mathematical ideas in historical context and shows how – for example – arithmetic leads to accountancy leads to commerce. Reminiscent in some ways of my own “Fifty Things That Made The Modern Economy”, except that the things in question are not fertiliser or the elevator or the bar code, but arithmetic and calculus and statistics.

Speaking of which – the sequel to “Fifty Things” is imminent in paperback…

…The paperback of “The Next 50 Things That Made The Modern Economy” is FINALLY coming – 26 August 2021. Please think about pre-ordering, which is hugely helpful in stimulating bookshops to stock and display the book.

“Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”- Bill Bryson

“Harford is a fine, perceptive writer, and an effortless explainer of tricky concepts. His book teems with good things, and will expand the mind of anyone lucky enough to read it.”- The Daily Mail

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

The strange temptations of phoney medicine

We puny humans just can’t seem to deal with the idea of a disease for which there is no treatment. We’ll always find something to believe in, no matter how tenuous. Since the Sars-Cov-2 virus was discovered, people have been circulating “cures”, from avoiding iced drinks (nope) to using special red soap (soap is good, its colour irrelevant).

Some speculative treatments have been pushed by politicians. The UK’s former Brexit supremo David Davis has urged the use of high-dose vitamin D supplements, while Donald Trump advocated hydroxychloroquine. Alas, a high-quality randomised trial has shown that hydroxychloroquine is not an effective treatment. Low-dose vitamin D is a useful supplement to take in winter, but there is no good evidence that high doses can treat Covid-19.

Then there is the pure quackery: inhaling hydrogen peroxide, using a USB flash drive as a “bioshield”, or drinking a suspension of silver particles in water. There is a lot of money to be made by selling snake oil to the fearful or desperate.

Phoney treatments are not new. Indeed, their enduring popularity is a puzzle. In 2010, the economist Werner Troesken published a study of the US market for unproven concoctions — “patent medicines” — in the 19th and early 20th centuries. He concluded that, even after adjusting for inflation, people spent more than a hundred times as much on such medicines in 1939 as they had back in 1810. This growth outpaced overall spending twenty times over, and the market for patent medicines eventually rivalled major industries such as fertilisers or paints.

So why the demand? Steven Johnson, in his new book Extra Life, points to two powerful forces helping to make useless treatments look good: the immune system and the placebo effect. The human immune system is a wonder: most people who are sick recover.

Now stir in the placebo effect, the well-known but poorly understood tendency of people to benefit simply from the belief they are being treated. Troesken notes that many quack medicines included ingredients such as chilli, alcohol and opium, producing plausible highs, lows and tingles. If you happen to recover from sore knees after applying petroleum oil seasoned with chilli pepper, turpentine and camphor — Clark Stanley’s Snake Oil Liniment — then you might well give Stanley the benefit of the doubt.

Chronically sick people rarely expected miracles, but were willing to give these treatments a try. What did they have to lose? The emergence of a few genuinely effective medical treatments did nothing to damp demand: it enhanced the plausibility of true and false medicines alike. So people hopped from potion to potion, more in hope than expectation. If these treatments had worked, writes Troesken, “people would have been cured . . . and spending would have been curtailed”. The fact that they did not cure much was, ironically, part of the reason they were in demand.

And where were the regulators? Johnson describes an irony: they finally took an interest because of a product that was not a quack medicine but a genuine antibiotic, sulphanilamide. Because it was not soluble in water or alcohol it was delivered as a chunky pill, difficult for children to swallow.

In 1937 a child-friendly version was marketed by a new drugs company, SE Massengill. It was a raspberry-flavoured syrup with the sulphanilamide dissolved in diethylene glycol. Too few people knew then that the glycol was toxic and there was no legal requirement to check. More than a hundred people, 34 of them children, died of kidney failure before the US Food and Drug Administration figured out the problem and tracked down the last poisonous bottle. Harold Watkins, who originally produced the deadly syrup, killed himself.

The FDA’s main role had been to ensure that the syrup did, as advertised, taste of raspberries. In 1938, President Franklin D Roosevelt signed a new law mandating that the FDA could also check for toxicity. And Frances Oldham Kelsey, the young chemist who identified the toxin in Massengill’s “Elixir”, went on to a senior role at the FDA, where she later refused to approve the use of the deadly drug thalidomide.

We consumers need help. We cannot distinguish tricks from treatments without high quality information. That information can come from public, profit-seeking or non-profit sources, and regulators are well placed to require the expensive checks needed.

Yet it is not easy to draw the line on what to regulate and how. The FDA, understandably, wants to avoid approving future child-killing drugs, but there is a price to be paid for caution. The FDA was slow to approve Covid tests in early 2020. It has seemed sceptical of the Oxford/AstraZeneca vaccine approved by British and European regulators and in use in more than 160 countries around the world. There is a risk in hasty approval but also a risk in delay.

I cannot help but draw a broader political lesson from the long history of demand for quackery. When we feel that things are not going well and that experts have been unable to help, we seek more speculative remedies, even if we do not expect much. When those remedies fail, we become more desperate, and we keep searching. There is always another charlatan around the corner.

Troesken underlines the puzzle, writing “one does not normally expect strong consumer demand for products that routinely fail to deliver”. Not strong voter demand, either. And yet here we are.

Written for and first published in the Financial Times on 27 June 2021.

The paperback of “How To Make The World Add Up” is now out. US title: “The Data Detective”.

“One of the most wonderful collections of stories that I have read in a long time… fascinating.”- Steve Levitt (Freakonomics)

“If you aren’t in love with stats before reading this book, you will be by the time you’re done.”- Caroline Criado Perez (Invisible Women)

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

The Tyranny of Spreadsheets

Early last October my phone rang. On the line was a researcher calling from Today, the BBC’s agenda-setting morning radio programme. She told me that something strange had happened, and she hoped I might be able to explain it. Nearly 16,000 positive Covid cases had disappeared completely from the UK’s contact tracing system. These were 16,000 people who should have been warned they were infected and a danger to others, 16,000 cases contact tracers should have been running down to figure out where the infected went, who they met and who else might be at risk. None of which was happening.

Why had the cases disappeared? Apparently, Microsoft Excel had run out of numbers.

It was an astonishing story that would, in time, lead me to delve into the history of accountancy, epidemiology and vaccination, discuss file formatting with Microsoft’s founder, Bill Gates, and even trace the aftershocks of the collapse of Enron. But above all, it was a story that would teach me about the way we take numbers for granted.

Now, as the UK tentatively reopens against a background of rapidly rising cases, we are hoping that vaccinations will keep us safe. The vaccines have — rightly — been trumpeted as a scientific triumph. Their development and rollout have taken place on a heroic scale.

But back in September and October, when the UK was also reopening against a strikingly similar backdrop of rising cases, we had no vaccine to protect us. Instead, we were trying to defend ourselves with data. And we didn’t seem to be nearly as enamoured of data as we now are of vaccines. That is a shame, because when you’re relying on numbers to keep you safe, it’s important to put some effort into keeping your numbers straight.

*

The origin of Excel can be traced back far further than that of Microsoft. In the late 1300s, the need for a solid system for accounts was evident in the outbursts of one man in particular, an Italian textile merchant named Francesco di Marco Datini. Poor Datini was surrounded by fools.

“You cannot see a crow in a bowlful of milk!” he berated one associate. “You could lose your way from your nose to your mouth!” he chided another.

Iris Origo’s vivid book The Merchant of Prato describes Datini’s everyday life and explains his problem: keeping track of everything in a complicated world. By the end of the 14th century, merchants such as Datini had progressed from mere travelling salesmen able to keep track of profits by patting their purses. They were now in charge of sophisticated operations.

Datini, for example, ordered wool from the island of Mallorca two years before the sheep had even grown it, a hedge to account for the numerous subcontractors that would process it before it became beautiful rolls of dyed cloth. The supply chain between shepherd and consumer stretched across Barcelona, Pisa, Venice, Valencia, North Africa and back to Mallorca. It took four years between the initial order of wool and the final sale of cloth. No wonder Datini insisted on absolute clarity about where his product was at any moment, not to mention his money.

How did he manage? Spreadsheets. Datini, of course, did not use Excel back in 1396. But he did use its direct predecessor: sheets of paper laid out according to the system of double-entry bookkeeping, otherwise known as bookkeeping alla veneziana. In double-entry bookkeeping, every entry is made twice. (The clue’s in the name.) For example, if you spend 100 florins on wool, that is recorded as a credit of 100 florins in your cash account and a debit of 100 florins worth of wool in your assets account. This extra effort of booking everything twice makes it much easier to detect mistakes. If one has been made, the books won’t balance.

Double-entry bookkeeping became an essential method for keeping track of who owed what to whom, foreign exchange transactions, profits, losses, everything. It helped Datini and merchants like him ensure nothing was lost, no matter how incompetent their associates.

A century later, the master of double-entry booking was Luca Pacioli. He was a serious mathematician and a friend of Leonardo da Vinci. But he’s best known today as the most famous accountant who ever lived. He literally wrote the book on the double-entry method, back in 1494. Pacioli once advised, “If you cannot be a good accountant, you will grope your way forward like a blind man and may meet great losses.”

We don’t have to accept Pacioli’s insensitive simile to understand his point: life is easier when you can see the obstacles and opportunities around you. Good accounts show us clearly what would otherwise be invisible. But if you can’t keep your spreadsheets straight, you may meet great losses. (More on that shortly.)

Nearly five hundred years later, in 1978, a student named Dan Bricklin sat in a classroom at Harvard Business School. As he watched his accounting professor filling in rows and columns on the blackboard, an idea dawned on him. Each time the professor made a change he would have to work across and down the grid, erasing and rewriting other numbers to make everything add up. Bricklin knew that this erasing and rewriting was happening every day, millions of times a day, all over the world, as accounting clerks adjusted the entries in what they called spreadsheets: big sheets of paper spread across two pages of an accounting ledger.

Bricklin was a geek and former programmer who immediately thought, “I can do this on a computer.” As Steven Levy described in a classic mid-1980s feature in Harper’s, the rest was history. Bricklin and a friend called their spreadsheet program VisiCalc. It went on sale on October 17 1979. It was a smash hit soon followed by Lotus 1-2-3 and then, in due course, by Excel.

For accountants, digital spreadsheets were revolutionary, replacing hours of painstaking work with a few taps on a keyboard. But some things didn’t change. Accountants still had their professional training and their double-entry system. The rest of us did not, but that did not prevent Excel from becoming ubiquitous. It was, after all, easily accessible and flexible, a tool like a Swiss Army knife for numbers, sitting in your digital back pocket. Any idiot could use it. And goodness, we did.

*

Nobody really knows what happened to the 16,000 positive Covid cases that disappeared from the spreadsheet. Public Health England (PHE), a government agency responsible for the process, still hasn’t published anything very informative on the issue.

“The suggestion that any cases were ‘lost’ is simply incorrect,” they told me. “No cases were missed. There was a delay in referring cases for contact tracing and reporting them in the national figures.”

That delay was typically four or five days, long enough to render the test result almost useless. If I mislaid my passport just before a holiday and then found it after five days staying at home instead, I am not sure I would triumphantly wave it in the air and declare, “The suggestion that my passport was ‘lost’ is simply incorrect.”

For a contact-tracing system, lost for five days is lost. The question is, how were they lost? Somewhere in PHE’s data pipeline, someone had used the wrong Excel file format, XLS rather than the more recent XLSX. And XLS spreadsheets simply don’t have that many rows: 2 to the power of 16, about 64,000. This meant that during some automated process, cases had vanished off the bottom of the spreadsheet, and nobody had noticed.

Everyone could see the funny side of the mishap. The idea of simply running out of space to put the numbers was darkly amusing. The fact that Microsoft was never anyone’s idea of cool simply added to the absurdity. Clippy, the maligned automated assistant from Office 2000, began making the rounds as a meme: “It looks like you’re trying to track a global pandemic. Would you like help?”

A few weeks after the data-loss scandal, I found myself able to ask Bill Gates himself about what had happened. Gates no longer runs Microsoft, and I was interviewing him about vaccines for a BBC programme called How to Vaccinate The World. But the opportunity to have a bit of fun quizzing him about XLS and XLSX was too good to pass up.

I expressed the question in the nerdiest way possible, and Gates’s response was so strait-laced I had to smile: “I guess… they overran the 64,000 limit, which is not there in the new format, so…” Well, indeed. Gates then added, “It’s good to have people double-check things, and I’m sorry that happened.”

Exactly how the outdated XLS format came to be used is unclear. PHE sent me an explanation, but it was rather vague. I didn’t understand it, so I showed it to some members of Eusprig, the European Spreadsheet Risks Group. They spend their lives analysing what happens when spreadsheets go rogue. They’re my kind of people. But they didn’t understand what PHE had told me, either. It was all a little light on detail.

They agreed that the basic problem was that whatever PHE had done wrong, it didn’t have the right checks and controls to flag problems. Or as Gates put it, “It’s good to have people double-check things.”

*

The original paper spreadsheets were designed to help us not lose our way, and one might naturally imagine the digital spreadsheet is not only faster but more accurate. Is it? One clue comes from a wonderful study conducted by Felienne Hermans, a computer scientist. A few years ago, Hermans realised that there was a bountiful source of spreadsheets she could study. That source was Enron, the bankrupt energy company.

After Enron collapsed in 2001 amid an epic accounting scandal, regulators extracted a cache of half a million emails from the company’s servers. Those emails are now publicly available and have been studied by researchers trying to understand everything from the evolution of informal written language to the way people use email folders. Hermans was interested in what was attached to some of these emails: spreadsheets.

She started digging through them, not looking for fraud, but for spreadsheets with obvious errors such as missing or circular references. Looking at nearly 10,000 spreadsheets with calculations in them, she found that a quarter had at least one such error. The errors even seemed to multiply. If a spreadsheet had any mistakes at all, on average it contained more than 750.

How can a spreadsheet acquire so many errors? Matt Parker, the author of Humble Pi, a book about mathematical mishaps and their consequences, notes that Excel’s own functionality combined with the mistaken assumptions of users will often introduce mistakes.

Type an international phone number into Excel, for example, and the program strips off the leading zeroes, which are redundant in a mathematical integer but not in a phone number. If instead you type in a twenty digit serial number, Excel will decide those 20 digits are a huge quantity and round them off, turning the last few digits into zeroes.

Or say you’re a genetics researcher typing in the name of a gene such as “Membrane Associated Ring-CH-Type Finger 1”, or March1 for short, or perhaps the Sept1 gene. You can imagine what Excel does next. It turns those gene names into dates. One study estimated that 20 per cent of all genetics papers had errors caused by Excel’s autocorrect.

Microsoft’s defence is simple enough: the default settings are intended to work in everyday scenarios. Which is the polite way of saying: Guys, Excel wasn’t designed for genetics researchers. It was designed for accountants.

But it’s understandable that scientists picked up Excel and started to use it. It’s powerful, it’s flexible. It’s ubiquitous. It may not be the right tool, but it’s the tool that’s right there.

When used by a trained accountant to carry out double-entry bookkeeping, a long-established system with inbuilt error detection, Excel is a perfectly professional tool. But when pressed into service by genetics researchers or contact tracers, it’s like using your Swiss Army Knife to fit a kitchen because it’s the tool you have closest at hand. Not impossible, but hardly advisable.

And yet when the genetics research community were wrestling with the autocorrecting genes issue, they resigned themselves to the hard truth that they would never wean people off Excel. Instead, the folks in charge — the Hugo Gene Nomenclature Committee — decided to change the names of the genes in question.

The decision is understandable. But it also neatly illustrates the contortions we go through as a result of treating data as an afterthought, just something to slap together on a spreadsheet. That is a shame, because history suggests that well-managed information can be transformative.

*

A few months ago, I asked folks on Twitter if they could recommend some good books about the eradication of smallpox. Most people recommended books about Edward Jenner, who in 1796 was the first to demonstrate an effective smallpox vaccine. That’s revealing, because I’d asked about the eradication of smallpox, and smallpox wasn’t eradicated in 1796. Not even close.

While eradication would have been impossible without a highly effective vaccine, it also required the highly effective use of information. Or as Datini might have said, it required not losing your way from your nose to your mouth.

Ever since the vaccine for smallpox was demonstrated in 1796, people dreamed of eradicating the disease. But those dreams kept failing to come true. In trying to vaccinate the entire planet, over and over again, the vaccinators never managed to reach quite enough people. In poorer countries, smallpox lingered in rural areas or neglected communities. A generation of babies were born without any immunity and, soon enough, the disease returned.

In the mid-1960s, smallpox was still killing two million people a year. The World Health Organization announced it would redouble its efforts to eradicate the disease and planned to do so by intensifying the mass vaccination campaign. One of those leading these efforts was Bill Foege, an Iowan-born epidemiologist who knew smallpox so well he could detect cases by smell. (Lesion-blistered skin has a distinctive odour.)

Foege would show up in a village in eastern Nigeria, all six foot seven of him, and the elders would put out the word, Come and see the tallest man in the world! And people did. Foege reckons he once vaccinated 11,600 people in a single day. It wasn’t enough to quash periodic outbreaks.

Then, late in 1966, Foege received a radio message warning of an outbreak of smallpox in a village about a hundred miles away. He travelled there, found five cases and vaccinated everyone they’d been in contact with. (The smallpox vaccine can still work even if it is given a day or more after people have been exposed to the virus.)

Standard practice then would be to vaccinate everyone for miles around. But Foege’s team just didn’t have enough doses. Instead, he used radio and the local network of missionaries to spot new cases. Every evening at seven o’clock, they’d switch on the radio and put the word out. Whenever an outbreak was reported, Foege and his team quickly raced to the scene and administered vaccines.

The hope was to create something like a firebreak, keeping the disease from spreading. And it worked. Using this tactic, Foege’s team eliminated smallpox from eastern Nigeria within six months. It was 1967, and soon civil war engulfed the country. Despite the chaos and enormous bloodshed of that war, smallpox did not return.

The secret was to worry less about the blanket coverage that was never quite good enough and to worry more about quickly finding exactly where each outbreak had appeared. Eradication was all about information. Up until that point, information had been very patchy. The WHO realised it had been finding only 100,000 or so cases each year against a backdrop of 10 million.

Foege’s experience showed that public health workers could beat smallpox if they had the data. The strategy became known as ring vaccination. It’s not the same as contact tracing, but it has a lot in common: in both cases you need to rapidly isolate infected people and find their recent contacts.

Ring vaccination worked. In less than a decade, doctors were scrambling to get to an outbreak in India so that they could observe a case of smallpox before the virus went extinct. The last gasp of smallpox in the wild was in Somalia, late in 1977. Ali Maow Maalin, 23 years old, a cook and part-time vaccinator, astonishingly, had not been vaccinated. He developed smallpox symptoms, was vaccinated — along with 91 friends and contacts — and recovered. Maalin devoted his life to the eradication of polio.

The vaccines were important. Essential, in fact. But so was quickly identifying and tracing contacts at risk. Smallpox had survived nearly two centuries of vaccinations — but it could not survive a well-run system that targeted outbreaks and tracked potential cases.

With hindsight, it seems simple. In a way, it was. But of course, keeping track of things is harder than it might first appear. Francesco di Marco Datini could have told you that.

*

One of the striking lessons of the pandemic has been how powerful data can be when handled well — and how much damage is done when the data are fumbled. Almost every question we have asked about this virus requires the deft use of statistics to answer it. Who has it? How does it spread? Who is most at risk? How can we treat it? Without a flow of good data and reliable ways to analyse that data, we haven’t a hope of answering such questions.

This is not just a case of having the right boffins solve the right equations. Data do not grow on trees: they must be assembled. An example of this process done right is Recovery (Randomised Evaluations of Covid-19 Therapy). Recovery is a system for running simple but powerful randomised trials of different Covid therapies as an integrated part of the regular treatment of hospital patients with Covid, all over the UK. It was set up at the start of the pandemic in a matter of days by two Oxford academics, Peter Horby and Martin Landray.

Recovery has produced a steady stream of vital findings, notably that the antimalarial drug hydroxychloroquine does not help and the cheap steroid dexamethasone is a lifesaver. (How many lives it has saved is unclear, but it’s surely well over a million by now.) It is an example of what can be done when we take seriously not only the data, but the “data infrastructure”, the tools and the processes we have to collect, manage and analyse that data.

It is hard to think of a clearer contrast with the misfiring contact-tracing systems in many supposedly sophisticated western democracies. Nature reported late last year that Australia, Washington state and Hawaii were still using phones or faxes to share information about new cases, and that public health professionals from Africa were aghast at the failure of the US system to learn the hard-won lessons of the Ebola outbreak.

There is more to running a good contact-tracing system than data infrastructure. But without good data the task is all but impossible. As with smallpox, success begins with rapidly figuring out where the virus is — and therefore, where it might go next.

Nor has the vaccine rendered contact tracing obsolete. Most people still aren’t vaccinated, and some people never will be. One day there will be another pandemic, and another, and another. We can’t guarantee vaccines will work every time, and vaccines take time to develop. While we wait, there will always be contact tracing. And good contact tracing, like thousands of other good things we want to achieve, requires investing in serious data infrastructure.

*

Let’s say you really want proof that contact tracing works, how would you get it?

Let’s also say you’re a mad scientist, crazed with power and unhindered by conventional ethics. You’d probably hack into the country’s contact-tracing system, then you’d delete some of the positive cases, making sure that some regions lost a lot of cases and some lost very few. This nefarious experiment would allow you to compare what happened in the places where the contact-tracing system was still running smoothly with the places where thousands of cases had gone missing.

If you weren’t an evil genius, of course, you wouldn’t dream of doing such a thing. Instead, you’d keep an eye out for it happening by accident because somebody bungled the formatting of Excel spreadsheets. Two economists, Thiemo Fetzer and Thomas Graeber, did just that. They decided that no catastrophe should be allowed to occur without trying to learn some lessons. They combed through the evidence from Public Health England’s mishap. And by comparing the experiences of different regions, they concluded that the error had led to 125,000 additional infections.

The story about Excel running out of numbers just seemed so bizarre at first. That’s why we were sharing Clippy memes, and why I took pleasure in teasing Gates about it. But his response, which seemed po-faced at the time, was right. He wasn’t laughing, because he understood that this wasn’t a comedy; it was a tragedy.

Fetzer and Graeber have calculated a conservative estimate of the number of people who died, unknown victims of the spreadsheet error. They think the death toll is at least 1,500 people.

So the next time there’s a pandemic, let’s make sure we have our spreadsheets in order. After all, as Luca Pacioli, the father of accounting, warned us more than five hundred years ago, without a good spreadsheet, you will grope your way forward, “and you may meet great losses”.

One thousand five hundred deaths. Relative to the scale of the whole pandemic, this is just a sliver of the total tragedy. But as the needless price of bad data management, they are great losses indeed.

This essay is adapted from an episode of my podcast, “Cautionary Tales“. It was published in the Financial Times on 24 June 2021.

The paperback of “How To Make The World Add Up” is now out. US title: “The Data Detective”.

“One of the most wonderful collections of stories that I have read in a long time… fascinating.”- Steve Levitt (Freakonomics)

“If you aren’t in love with stats before reading this book, you will be by the time you’re done.”- Caroline Criado Perez (Invisible Women)

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

Aiming for the moon, Hard Times, and Marlon and Jake read dead people

I was delighted that the More or Less team was awarded the Royal Statistical Society’s excellence in journalism award for coronavirus reporting, and also very pleased to have been highly commended in the category of specialist journalist in the UK Press Awards. Congratulations to all the worthy winners, many of whom I am lucky to be able to count as colleagues.

My latest podcast recommendation is Marlon and Jake Read Dead People – the almost unbelievably erudite Marlon James and Jake Morrissey argue in suitably opinionated fashion about their favourite books by dead authors.

If you think there’s anything more disheartening as an author than hearing an editor rip into (of all people) Ernest Hemmingway for time-wasting, try this: the editor in question is YOUR EDITOR. I suppose the next time I send Jake a manuscript and get some tough notes back, I can at least console myself that he was even harder on Hemmingway.

Jake has set Marlon reading James Thurber’s short humorous memoir My Life and Hard Times (Kindle, but also available free online). I thought I’d have a try and it is indeed highly amusing.

Meanwhile you might want to check out my conservation with Mattie Henry and Taylor Bledsoe, the two young hosts of the “Aiming for the Moon” podcast.

The paperback of “The Next 50 Things That Made The Modern Economy” is FINALLY coming – 26 August 2021. Please think about pre-ordering, which is hugely helpful in stimulating bookshops to stock and display the book.

“Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”- Bill Bryson

“Harford is a fine, perceptive writer, and an effortless explainer of tricky concepts. His book teems with good things, and will expand the mind of anyone lucky enough to read it.”- The Daily Mail

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

Why we lose track of spending in a cashless society

What looks like fraud, feels like fraud but isn’t fraud? What about a company website that pops up when you search for the government agency that issues driving licences, and charges a handsome fee for forwarding your details to the real website? Personal finance campaigners have been complaining about such sites for years, but I think there is a broader lesson to be drawn about the way we spend our money these days.

Between outright fraud and honest commerce there may be a sharp legal line — but economically and psychologically the distinction is a gradual blur. I worry that we now live in that blur, spending cash without clearly perceiving what happened.

The pandemic, with its shift to contactless or online spending, has served to catalyse the process further. Consider the scam-adjacent website: the principal service it provides is to solve a problem it has paid to create — namely that the official website isn’t as prominent in web searches as it might be. The near-valueless service, however, is described unambiguously and the website states plainly that it is not an official site.

So who would knowingly use such a service? My wife recently did. And it is self-evident that my wife is a person of the most refined wisdom and discernment. So how did this calamity occur? My wife handed over about £100 for nothing of value because her technology made it so easy. She didn’t notice the disclaimer because she was distracted and multitasking. She applied via a mobile phone, her browser preloaded with credit card details. On such a tiny screen, hints of trouble pass unseen.

We’re warned to be careful of the small print, but on a mobile phone, all the print is small print.

This is why I argue such websites exist on a continuum. There’s the outright fraudster who tries to panic you into sending thousands of pounds to avoid ruin or prison. Then there’s a blue-chip website such as Amazon, which undoubtedly provides a real service, but would be delighted to make it easier for customers to spend impulsively.

Amazon famously secured an absurd patent on “one-click” online retail ordering in 1999. (Steve Jobs, with typical foresight, immediately licensed it for Apple for a miserly million dollars.) That patent has now expired, but Amazon is still keen to make spending effortless.

Every time I use my phone to check the sales rank of my book on the Amazon website — roughly every 27 minutes — Amazon urges me to download its app. I don’t doubt that the app would work better, which is why I don’t want it.

In their book Nudge, Richard Thaler and Cass Sunstein urge regulators and companies to “make it easy” for people to do the right thing — such as pay taxes, sign up as an organ donor or save for a pension. As Thaler and Sunstein well know, nudges can also be used to “make it easy” to do other things, too — such as send money to a near-valueless licence-application service.

The logical extreme is the endlessly renewable subscription. Alongside the familiar bills for utilities, internet, mobile phone and mortgage, our household subscriptions include services as varied as an online yoga resource, access to all the Star Wars and Marvel movies, a Patreon campaign, wine, Amazon Prime, Microsoft Office, Adobe Photoshop, apps for mindfulness, language learning and productivity, two cloud storage services, unfettered access to BoardgameArena and a music bot on Discord.

Some of that will be incomprehensible, I’m sure; 15 years ago it would have been not just incomprehensible, but unimaginable. Yet not only are we paying for all this, we’re paying without a clear idea of when or how much the payments are, or even the method of payment we are using.

In a classic article from 2006, “Paying Not To Go To The Gym”, economists Stefano DellaVigna and Ulrike Malmendier compared consumers paying for health club membership in three different ways: with a 10-visit pass, on an annual membership and with an auto-renewing monthly subscription. The monthly consumers had more flexibility — and paid for the privilege — but they did not use it. Instead, they stayed subscribed for longer, paid nearly twice as much per gym visit and typically took more than two months to cancel after their final gym appearance. All these online subscriptions are plugging into something that health-club owners have known all along.

So what should we do? On the FT Money Clinic podcast, I recently advised a listener who felt guilty that she was spending impulsively online, and often regretted and returned the purchases. One suggestion I had for her was that instead of buying immediately, she should instead write down each item on a spreadsheet, to revisit at the end of the month. She would have time to reflect, and she would also see the cumulative price tag for all her temptations.

My thinking was that by making the spending harder, slower and more conspicuous, she might gain some degree of control. It was only afterwards that I realised how directly this advice was swimming against the commercial tide. Harder, slower, more conspicuous? Companies have long wanted spending to be fast, easy and barely worth a thought. Thanks to the miracles of modern technology, they are closer than ever to realising their desires.

Written for and first published in the Financial Times on 18 June 2021.

The paperback of “How To Make The World Add Up” is now out. US title: “The Data Detective”.

“One of the most wonderful collections of stories that I have read in a long time… fascinating.”- Steve Levitt (Freakonomics)

“If you aren’t in love with stats before reading this book, you will be by the time you’re done.”- Caroline Criado Perez (Invisible Women)

I’ve set up a storefront on Bookshop in the United States and the United Kingdom – have a look and see all my recommendations; Bookshop is set up to support local independent retailers.

Why do we work so hard?, and other intriguing questions

The Possiblity Club podcast – I had a good chat to Richard Freeman at the Possibility Club podcast – listen in if you wish! Brief Q&A also with the Histocrats.

Why do we work so hard? I was fascinated by James Suzman’s interview with Ezra Klein recently. He said a couple of very odd things (apparently labour income is no longer relevant, it’s all capital income?) but was enormously thought-provoking about low-work civilisations, high-work civilisations, the importance of culture, the city as the engine of desire and social comparison. All in all it made me eager to seek out his book, “Work”.

I’m speaking at the Budleigh Salterton Literary Festival in mid September – hopefully plenty of time for everyone to get fully vaccinated. Come along!

On my pile: Rationality by Steven Pinker. Self-recommending. The book is out late September.

Finally, I was delighted that How To Make The World Add Up is yet again the number one business bestseller on the monthly Sunday Times list. That’s December, May and June. If you’re one of the people who contributed to that success by buying a copy or spreading the work – thank you.