Humble Pi: When Math Goes Wrong in the Real World

by Matt Parker

So we now know that the clocks going forward and back does not increase the number of heart attacks (but rather, a lack of sleep can bring on a heart attack that would have happened anyway).

A repair mission was flown to the space telescope to add in corrective optics. A space-telescope contact lens, of sorts.

Many systems are accurate enough most of the time but break in “edge cases” where errors can be amplified. An app that points toward Mecca has to know where both the phone and Mecca are, but only to a low degree of accuracy to point in the right direction from most places on the planet. Until the phone is held right next to the Kaaba (a building at the center of Islam’s most important mosque).

The Swiss cheese model looks at how “defenses, barriers, and safeguards may be penetrated by an accident trajectory.” This accident trajectory imagines accidents as similar to a barrage of stones being thrown at a system: only the ones that make it all the way through result in a disaster. Within the system are multiple layers, each with its own defenses and safeguards to slow mistakes. But each layer has holes. They are like slices of Swiss cheese.

I love this view of accident management, because it acknowledges that people will inevitably make mistakes a certain percentage of the time. The pragmatic approach is to acknowledge this and build a system robust enough to filter mistakes out before they become disasters. When a disaster occurs, it is a system-wide failure, and it may not be fair to find a single human to take the blame.

As an armchair expert, it seems to me that the disciplines of engineering and aviation are pretty good at this. When researching this book, I read a lot of accident reports, and they were generally good at looking at the whole system. It is my uninformed impression that in some industries, such as medicine and finance, which do tend to blame the individua...

If humans are going to continue to engineer things beyond what we can perceive, then we need to also use the same intelligence to build systems that allow them to be used and maintained by actual humans. Or, to put it another way, if the bolts are too similar to tell apart, write the product number on them.

Thankfully, this was way back in 2015, and in the years since then, people have become much better at spotting fake news online.

Arguably, the need to crunch census data led to our modern computing industry.

But how many of the 4,063 people in the survey could wear such an approximately average uniform? The answer was 0. No member of the entire 4,063-people survey was in the middle 30 percent for all ten possible uniform measurements. The tendency to think in terms of the “average man” is a pitfall into which many persons blunder when attempting to apply human-body-size data to design problems. Actually, it is virtually impossible to find an “average man” in the air force population. This is not because of any unique traits of this group of men, but because of the great variability of bodily dimensions which is characteristic of all men. —The “Average Man”?, Gilbert S. Daniels

Just because something survives does not mean it is significant.

I find that a lot of nostalgia about how things were manufactured better in the past comes down to survivor bias. I see people online sharing pictures of old kitchen equipment that is still working: waffle irons from the 1920s, mixers from the 1940s, and coffee machines from the 1980s. And there is some truth to the statement that older appliances last longer. I spoke to a manufacturing engineer in the US who said that, with 3D-design software, parts can be designed with much smaller tolerances now, whereas previous generations of engineers were not sure where the line was and so had to over-engineer parts to make sure they would work. But there is also the survivor bias, the fact that all the kitchen mixers that broke over the years have long been thrown out.

A negative result from a drug trial is twice as likely to remain unpublished as a positive result.

It is a mathematical certainty that you can find any pattern you want, as long as you’re prepared to ignore enough data that does not match.

Between 1993 and 2008, the police in Germany were searching for the mysterious “phantom of Heilbronn,” a woman who had been linked to forty crimes, including six murders; her DNA had been found at all the crime scenes. Tens of thousands of police hours were spent looking for Germany’s “most dangerous woman,” and there was a €300,000 bounty on her head. It turns out she was a woman who worked in the factory that made the cotton swabs used to collect DNA evidence.

Otherwise, when you hear a statistic such as the fact that cancer rates have been steadily increasing, you could assume that people are living less healthy lives. The opposite is true: longevity is increasing, which means more people are living long enough to get cancer. For most cancers, age is the biggest risk factor and, in the US, 77 percent of all cancer diagnoses are for people aged fifty-five or older. As much as it pains me to say it, when it comes to statistics, the numbers are not everything.

You can’t have a line of code saying do something random and get a truly random number without a specialized component being attached to the computer.

In the Department of Quantum Sciences at ANU, they have a detector listening to a vacuum, converting the quantum foam into random numbers, and then streaming them live at https://qrng.anu.edu.au, around the clock. For tech people, they have a great range of secure delivery systems (never use the built-in random.random() function in the Python programming language again!).

But sadly, those numbers are not truly random. Like the Big Board, they are following a predetermined sequence of values. Except, instead of memorizing a list in advance, they generate it on the fly. Pseudorandom-number generators use mathematical equations to generate numbers that have all the hallmarks of being random but are just pretending to be.

When searching around for a seed to generate random numbers, Netscape would use a combination of the current time and its process identifiers. On most operating systems, whenever a program is running it is given a process ID number so your computer can keep track of it. Netscape would use the process ID of the current session as well as the process ID of the parent program that opened Netscape, combined with the current time (seconds and microseconds) to seed its pseudorandom-number generator.

In 1995 Ian Goldberg and David Wagner (then computer science PhD students at the University of California–Berkeley) showed that a clever malicious agent could produce a list of possible random seeds small enough that a computer could check them all in a matter of minutes, rendering the encryption useless.* Netscape had previously turned down offers of help from the security community but, after the work of Goldberg and Wagner, they patched the problem and released their solution to be independently scrutinized by anyone who wanted to go through it with a fine-code comb.

Because using an amazing pseudorandom-sequence generator with an easy-to-guess seed is like buying an expensive lock and then using it as a doorstop. Or indeed buying an expensive lock and leaving the screws visible and unscrewable.

One of the first standard methods for generating pseudorandom numbers was to multiply each number in your sequence by a large multiplier K then divide the answer by a different number M and keep the remainder as your next pseudorandom term. This was used by almost all early computers, until George Marsaglia, a mathematician at Boeing Scientific Research Laboratories, spotted a fatal flaw in 1968. If you took the sequence of random numbers coming out and plotted them as coordinates on a graph, they would line up. Admittedly, this could require complicated graphs with upward of ten dimensions.

Getting quality pseudorandom numbers continues to be a problem. In 2016 the Chrome browser had to fix its pseudorandom-number generator. Modern browsers are now pretty good at producing seeds for their pseudorandom numbers but, unbelievably, the generators themselves can still have problems. Chrome was using an algorithm called MWC1616, which was based on a combination of multiplication with carry (the MWC from the name) and concatenation to generate pseudorandom numbers. But it accidentally repeated itself, over and over. What a bore.

Some programmers had released a Chrome extension that people could download and use. To anonymously keep track of everyone who had installed it, upon installation it would generate a random number as an arbitrary user ID and send that back to the company’s database. They had a graph in the office showing a nice increase in installations of their extension until, one day, the number of new installs dropped to zero. Had the whole world suddenly decided to stop using their extension? Or was there some fatal flaw in their code that had caused it to stop working? No. Their extension was working fine, and people were still installing it. But it used the JavaScript programming language and called the built-in function Math.random() to get a new user ID number for each new install. This worked fine for the first few million cases but, from then on, it was returning only numbers that had already been used. This meant that all new users looked like duplicates of those already in the database. These user ID numbers were 256-bit values with possibilities measured in the quattuorvigintillions (around 1077). There is no way they should repeat so quickly. Something had gone wrong, and the MWC1616 algorithm was to blame. The pseudorandom numbers had looped. This was not the only case of Chrome users having random problems and, thankfully, the developers behind the browser’s JavaScript engine set about fixing the problem. As of 2016 Chrome has switched to an algorithm called xorshift128+ w...

If there is any moral to this story, it’s that, when you are writing code, remember that someone may have to comb through it and check everything when it is being repurposed in the future. It could even be you, long after you have forgotten the original logic behind the code. For this reason, programmers can leave “comments” in their code, which are little messages to anyone else who has to read their code. The programmer mantra should be “Always comment on your code.” And make the comments helpful.

Programming is such a great combination of complexity and absolute certainty. Any one line of code is completely defined: a computer will do exactly what the code says. But determining the end result of a lot of code interacting is rather difficult, and this can make debugging code an emotional experience.

I think, due to survivor bias, programmers tend to be a sadistic bunch who enjoy frustration.

This goes to show that, even though some sysadmins see themselves as gods on Earth, they still have to obey the laws of physics.

Keyboard error or no keyboard present Press F1 to continue, DEL to enter SETUP.

Programming has inarguably been a huge benefit to humankind, but it is still in its early days. Complex code will always react in ways its developers did not see coming. But there is the hope that well-programmed devices can add a few extra slices of cheese into our modern systems.

Because we all make mistakes. Relentlessly. And that is nothing to be feared. Many people I speak to say that, when they were at school, they were put off mathematics because they simply didn’t get it. But half the challenge of learning math is accepting that you may not be naturally good at it, but if you put the effort in, you can learn it.

The CHI+MED team who are researching the computer-human interactions with medical devices actually came up with a new version of the Swiss cheese model, which I’m quite partial to: the hot-cheese model of accident causation. This turns the Swiss cheese on its side: imagine that the slices of cheese are horizontal and mistakes are raining down from the top. Only mistakes that fall down through holes in every layer make it out the bottom to become accidents. The new element is that the cheese slices themselves are hot and parts of them are liable to drip down, causing new problems. Working with medical devices made the CHI+MED folks realize that there was a cause of accidents not represented by the Swiss cheese model: layers and steps within a system could themselves cause mistakes to happen. Adding a new layer does not automatically reduce how many accidents are happening. Systems are more complicated and dynamic than that.

No one wants extra drips in the fondue pot of disaster.

Having bar codes caused situations where the patients and drugs were less thoroughly checked than they were before. If a new system is implemented, humans can be very resourceful when finding new ways to make mistakes.