A Hacker's Mind: How the Powerful Bend Society's Rules, and How to Bend them Back

by Bruce Schneier

Like Club Penguin, many online games for children have tried to place restrictions on speech, to prevent bullying, harassment, and predators. Kids have hacked them all. Tricks to evade moderators and swear filters include deliberate misspellings like “phuq,” separating out key information over several utterances so that no single utterance breaks the rules, and acrostics. Some sites prohibited users from typing numbers; kids responded by using words: “won” for one, “too” for two, “tree” for three, and so on. Same with insults: “lose her” for loser, “stew putt” for stupid.

Algospeak

That hack wasn’t new. It even has a name: foldering. In separate incidents, it was used by General Petraeus, Paul Manafort, and the 9/11 terrorists. They all realized that they could evade communications surveillance if they shared an email account with their co-conspirators and wrote messages to each other, keeping them as email drafts and never sending them.

Collect calls had a hefty surcharge. But because the operator initiated the call, information could be transmitted before anything was charged. So, we would call collect, the operator asking the other party—generally our parents—if they would accept a collect call from us. Our parents would say no and then return the call at standard, less expensive, rates. This kind of thing could be made more efficient. Some families had a list of names to tell the operator; they were all coded messages of some sort: “Bruce” meant “arrived safely,” “Steve” meant “call back,” and so on. (The operator had no idea what the caller’s real name was.) Even today, people have phone hacks to get around billing rules.

Systems tend to be rigid and rule-bound. Systems limit what we can do, and, invariably, some of us want to do something else. So we hack. Once you’re familiar with what systems are and how they operate, you’ll see them everywhere. And then you’ll see hacking everywhere.

This doesn’t imply that all systems are broken. Recall Gödel. There’s a saying among lawyers: “All contracts are incomplete.” A contract works not because it rigidly prevents the parties from subverting its intent; it works because most loopholes are filled with trust and good intention—and there’s a system of arbitration and adjudication available if things go badly. It might sound naive and idealistic, but systems of trust are what make society work. We don’t demand airtight protection in our agreements, because (1) that’s impossible to achieve, (2) any attempt will be too long and unwieldy, and (3) we don’t really need it.

We rightly trust that most people don’t hack systems. And we have systems to deal with hacks when they occur. This is resilience. This is what makes society work. It’s how we humans have dealt with hacking for millennia.

Not all systems are equally hackable. As we move through the book, we’ll see various characteristics of systems that make them more or less vulnerable to hacking. Complex systems with many rules are particularly vulnerable, simply because there are more possibilities for unanticipated and unintended consequences. This is certainly true for computer systems

I’ve written in the past that complexity is the worst enemy of security—and it’s also true for systems like the tax code, financial regulations, and artificial intelligence. Human systems constrained by more flexible social norms and rules are more vulnerable to hacking, because they lea...

Hacking is a natural part of the human condition. It’s ubiquitous and, as we’ll see, an evolutionary process: constant, unending, and capable of creating, as Darwin would say, “forms most beautiful and most wonderful”—or most strange and terrible.

In 2011, an Australian bartender named Don Saunders figured out how to get free money from ATMs. He stumbled into the hack late one night. (It makes a better story if we imagine he was drunk at the time.) He noticed a way that he could transfer money he didn’t have from one account to another, then withdraw the cash without the system recording the transaction. The bonanza resulted from a vulnerability in the ATM’s software used to record transfers between accounts, combined with another vulnerability in the time lag in how the various accounts credited and debited when the ATMs went offline for the night. Saunders didn’t understand any of that, though. He found it by accident, and realized that he could reproduce the result.

Let’s pause for a second and talk about what’s being hacked here. Stealing money from a bank is always illegal. The hack isn’t of the banking system; the hack is of the ATM system and the bank’s software. Saunders found an unintended and unanticipated way to use those systems—to do things that the systems allowed—in a way that subverted their intent. That’s the hack.

The decades-long evolution of ATM attacks and resultant security countermeasures nicely illustrates the arms race between hackers and defenders. More than that, it illustrates several themes that we’ll return to throughout the book. Systems never exist in isolation. They’re made up of smaller systems, and they’re part of larger systems. ATMs are computer software, yes. But they’re also physical objects. Their use involves customers, and a remote banking network. Hackers can target any of those aspects of ATMs.

Another family of hacks involves stealing information in order to create and use a duplicate card. This is called “skimming,” and has become widespread and sophisticated over the years. The canonical hack involves placing a second magnetic-stripe reading device over a card slot, so that the customer unwittingly slides his card through the malicious reader along with the ATM’s real reader. Add a hidden camera or a sensor on the keypad, and a criminal can steal the PIN as well.

These hacks exploit several vulnerabilities. First, the customer doesn’t have enough expertise to notice a skimmer or a fake ATM. Second, a magnetic-stripe ATM card is easily duplicated. And third, the ATM authentication system—possession of the ATM card and knowledge of the PIN—just isn’t that secure.

Other ATM hacks target the software. In the hacking literature this is known as “jackpotting”: making the ATM spit bills out like coins from a slot machine, no stolen card or PIN required. A 2016 attack of this sort was hatched in Taiwan and then quickly spread across Asia, Europe, and Central America, resulting in losses in the tens of millions of dollars.

There’s no good data on how much money is stolen this way—banks are loath to make details of this sort of thing public—but the US Secret Service began warning financial institutions about jackpotting in 2018. And that’s eight years after security researcher Barnaby Jack demonstrated jackpotting at the DEF CON hacker conference in 2010. His attacks didn’t require anyone to physically tamper with the ATM; he found software vulnerabilities that he could remotely exploit to accomplish the same result.

Casinos have responded in two different ways. The first is to make card counting more difficult. Many casinos shuffle six decks of cards together—automatic shufflers do the work—and only deal two-thirds of the way through the decks to reduce the player’s probabilistic advantage. Or they shuffle after every hand. In both Las Vegas and Atlantic City, pit bosses are known to come around and engage suspected card counters in conversation to both distract and intimidate them.

Since casinos are private business, they generally can (depending on the state) deny service to whomever they want, as long as they don’t illegally discriminate while doing so.

One of the early hacks was called “mileage runs.” Miles, which fliers earn on the basis of the distance traveled, are basically a private currency that can be redeemed for tickets. A clever hacker will look for ways to arbitrage the two currencies: instances where you can get a lot of miles for not a lot of money. So, for example, flying nonstop from New York to Amsterdam is 3,630 miles, but connecting through Istanbul is 6,370 miles. If the two tickets cost the same, and you have nothing better to do with your time, that’s a great deal.

Other hacks involved ways to accrue points other than by flying. Airlines have long had affiliations with credit cards. These cards offer miles with every purchase, but often also large mileage bonuses when signing up. The hack is obvious: sign up for a lot of credit cards, and cancel before any fees accrue. One man opened a credit card and immediately bought $3,000 worth of Amazon gift cards to qualify for a sign-up bonus. Another filled his garage with blenders for a promotion offering extra points on appliances. A third boasted that she had “taken out over forty-six credit cards in five years and earned 2.6 million miles just in sign-up bonuses.”

The harm, of course, is that the banks end up paying billions of dollars in flights and other rewards for customers who are not paying fees or interest on their cards, and that these costs are passed on to consumers as higher ticket prices. Some credit cards have tried to clamp down on these hacks. In 2016, Chase instituted a rule that a consumer won’t be approved for most Chase credit cards if that person has opened five or more credit card accounts across all banks in the past twenty-four months.

American Express now revokes miles of people who have “engaged in abuse, misuse or gaming in connection with earning or using points,” giving it a broad ability to penaliz...

COVID-19 is a hacker. Like all viruses, SARS-CoV-2 is a clever exploitation of our body’s immune system, subverting the normal operation of that system at the expense of our general health and the lives of over 6 million people worldwide. HIV is another hacker. It infects T-helper white blood cells in our body, inserting its own DNA into the cell’s normal DNA, and then replicating inside the cell.

Eventually the infected cell releases more HIV into the bloodstream, continuing the multiplication process. In general, hacking is parasitical. Both HIV and SARS-CoV-2 are parasites: hosted by another species and benefiting from that arrangement, usually at the host’s expense. A system exists to further a set of goals, usually put forth by the system’s designers. A hacker hijacks the same system for a different set of goals, one that may be contrary to the original ones.

The idea of spam started in the 1990s, both in email and in the then-popular Usenet messaging service, and became a serious problem in the early 2000s. In those years, an estimated 90% of all email was spam. It’s a parasitical hack of a communication system.

Not all parasitical relationships come at the expense of the host, and not all hackers are evil. Usually they’re behaving rationally, in their own self-interest. They might be acting in their financial self-interest, like most of the examples in this book.

Like any parasite, hacking can’t be too effective at subverting a system; it needs the system to exist in order to work. So while ATM hacking can be a profitable criminal enterprise, it depends on there being ATMs to hack. If ATM hacking were too successful, banks would stop installing these oh-so-convenient cash machines.

hack that is too effective can end up making itself obsolete, by destroying the underlying system it depends on.

Spectre and Meltdown are two hardware vulnerabilities in Intel and other microprocessors; they were discovered in 2017 and announced in 2018.

Defending against hacks can be hard. Countermeasures range from patching to secure systems design, and we’ll talk about each of them in turn.

The first and most obvious defense is to remove the enabling vulnerability.

In the computer world, the primary defense against hacking is patching. It’s a straightforward technique: update the computer code to eliminate the vulnerability. No vulnerability, no exploit. No exploit, no hacking.

How well patching actually works depends a lot on the type of system we’re talking about. Systems that are owned or controlled by a singular entity can, if they want to—that is, if it makes economic sen...

Issuing the patch is just the first step of the process; next, the patch needs to be installed on the vulnerable systems. Historically, there has been a large disconnect between com...

This scenario assumes that the singular owning entity has the ability to write the patch and cares enough to do so, and that the system can be patched. If that company has enough engineers on staff to write patches, and if there is an update system to quickly push the new software out to every user, then patching can be a very effective security technique. If one of those two things doesn’t exist, then it isn’t. (Remember that there are many IoT devices whose code is in firmware and can’t be patched.)

Lots of high-profile hacks have occurred because of unpatched systems. China hacked Equifax in 2017 through a vulnerability in the Apache Struts web-application software. Apache patched the vulnerability in March; Equifax failed to promptly update its software and was successfully attacked in May.

Also in 2017, the WannaCry worm spread to over 200,000 computers worldwide and caused as much as $4 billion in damage, all to networks that hadn’t yet installed the patch for a Microsoft Windows vulnerability.

This illustrates a major downside of patching: it occurs after the fact. The vulnerability is already in the system. Hackers may be actively exploiting it at the time of the patch. And even if they aren’t, the very act of patching calls attention to the vulnera...

Large organizational networks have to deal with patching in a slower, cautious manner. Because a bad patch can cause all sorts of problems by the way it interacts with other critical software, patches are generally installed deliberately and methodically.

Patching works differently with social systems than with technological systems. With the latter, the patch makes the latest hack no longer possible. This is obviously true for software, but extends to other technological systems as well. ATM manufacturers can patch their machines so that a particular jackpotting hack simply doesn’t work anymore. A casino can deal blackjack from a six-deck shoe that continuously shuffles the cards. Financial exchanges can restrict trading to ten-second intervals, making hacks like high-frequency trading impossible.

With social, economic, or political systems that don’t directly involve computers, it’s not as clean. When we talk about “patching” the tax code or the rules of a game, what we mean is changing the laws or rules of the system so that a particular attack is no longer permitted.

So while it still might be possible to use a computer to predict roulette or to curve your hockey stick more than three-quarters of an inch, anyone caught doing it will experience the consequences. The only “installation” necessary is education: making sure that every casino pit boss and hockey referee knows the new rules and how to spot cheaters—and then punish them accordingly. Similarly, a legal tax avoidance strategy becomes illegal tax evasion, and is prosecuted if discovered (or so one would hope).

This points to another problem: cheaters can be tough to spot. Recall that roulette was vulnerable until the betting system was changed so that the hacks were no longer effective. This problem will come u...

Patching is also less effective when the governing body functions slowly, or when the governing body doesn’t have a unified vision of whether a patch is even necessary. That is, when the system doesn’t have a clear goal. What, for example, does it mean to “patch” the tax code? In most cases, it means passing another law that closes the vulnerabilities from the original law. That’s a process that can take years, because the tax code is created in the political realm, which is characterized by competing visions of what public policy should accomplish.

Also, the very people who take advantage of the vulnerability will attempt to hack the legislative systems to ensure that the law continues to permit their actions. Imagine if blackjack card counters were in charge of casino rules. Blackjack card counting would be celebrated as a clever, honest way to win the game in the same way tax avoidance is celebrated as smart.

In the absence of legislative patches, a court can quickly target a very specific patch. In the computer world, this is known as a hotfix: a fast software update designed to fix a particular bug or vulnerability. The term comes from the fact that, traditionally, these updates were applied to systems that were up and running: hence “hot.” It’s more risky; the software could crash, with whatever problems could result from that. Hotfixes are normal today—updates to your operating systems are applied while they are running, and a lot of stuff is running in the cloud—but when the term was coined, that wasn’t the case.

Reducing a hack’s effectiveness is a second defense.

Business email compromise is a social engineering attack, in that it exploits a vulnerability in people rather than a vulnerability in technology. In this scam, the victim receives an email from a normally trusted source, making a normally legitimate request but asking him or her to do it differently than usual, often against established protocol.

Sometimes email accounts of legitimate vendors are hacked in this scam, which increases the likelihood that the target will trust the sender. More often, the scam emails are slight variations of legitimate addresses: person@c0mpanyname.com instead of person@companyname.com, for example. (If you can’t tell or are listening to this as an audiobook, the “o” in “companyname” is actually a zero.) The vulnerability here is human inattentiveness, or misplaced trust.

When we can’t patch a vulnerability, we have three options. The first is to redesign the system so that the hack is too difficult, too expensive, less profitable, or generally less damaging. This also works when outlawing a hack isn’t enough, and we want to make it harder as well. The second is foreknowledge. If I can teach you about business email compromise and how it works, you will become better able to recognize when you are being targeted by it, and—hopefully—less likely to fall for it. This is how we defend against email and phone scams that slip through automated filters. This is how potential victims can resist effective “cognitive hacks” that play on universal human biases like fear and deference to authority. The final option is to employ an additional system to somehow secure the vulnerable system.