This Is How They Tell Me the World Ends: The Cyberweapons Arms Race

by Nicole Perlroth

For how long exactly, no one could predict; but between 2016 and 2017 the gap between the United States’ cyber capabilities and those of every single other nation and bad-faith actor on earth closed substantially. Starting in 2016, the U.S. National Security Agency’s own cyber arsenal—the sole reason the United States maintained its offensive advantage in cyberspace—was dribbled out online by a mysterious group whose identity remains unknown to this day. Over a period of nine months a cryptic hacker—or hackers; we still don’t know who the NSA’s torturers are—calling itself the Shadow Brokers started trickling out NSA hacking tools and code for any nation-state, cybercriminal, or terrorist to pick up and use in their own cyber crusades.

The documents were littered with NSA claims that the agency's hackers had access to nearly every piece of commercial hardware and software on the market. The agency appeared to have acquired a vast library of ways into every major app, social media platform, server, router, firewall, antivirus software, iPhone, Android, BlackBerry, computer, and operating system. In the hacking world, these invisible entry points have sci-fi names: they call them zero-days.

At the most basic level a zero-day is a software or hardware flaw for which there is no existing patch.

In the 1990s, the Pentagon’s military budgets were chopped by a third, with cyber being the one exception. Congress continued to approve vague “cybersecurity” budgets, without much grasp of how dollars funneled into offense or defense or even what cyber conflict necessarily entailed. Policymakers’ thinking on cyber conflict was, as former commander of U.S. Strategic Command James Ellis put it, “like the Rio Grande, a mile wide and an inch deep.” But inside each agency, officials were learning that the best zero-days netted the best intelligence, which in turn translated to bigger cyber budgets down the road.

In the beginning, Sabien’s team started scanning BugTraq, taking the bug discoveries that hackers were volunteering for free and tweaking them slightly before baking them into their own exploit. But eventually they started reaching out to hackers on the forums directly, inquiring whether they’d be willing to develop something unique for Sabien’s customers and never tell a soul. The money provided plenty of incentive. In the mid-1990s, government agencies paid contractors roughly $1 million for a set of ten zero-day exploits. Sabien’s team would budget half that to buy bugs and then develop them into exploits themselves. A decent bug in a widely used system like Windows might fetch $50,000; a bug in an obscure system used by a key adversary might fetch twice as much. A bug that allowed the government’s spies to burrow deep into an adversary’s system, undetected, and stay awhile? That might net a hacker $150,000.

When Deeley’s team searched back through their inventory, they discovered that the first implant had been installed in a typewriter shipped to the embassy in 1976. That meant that by the time Gunman was complete, the Soviets had been siphoning American secrets off their typewriters for eight years. Routine embassy inspections had missed the implants. American inspectors had come close after finding an antenna in the embassy chimney at one point, but they’d never discovered its purpose. And analysts never wondered why the Soviets treated their own typewriters with such paranoia. The Soviets banned their own staff from using electric typewriters for classified information, forcing them to use manual models for top-secret information. When Soviet typewriters weren’t being used at the country’s own embassies, the Soviets stored them in tamperproof containers. And yet the Americans hadn’t bothered to ask why.

I asked nearly every single one of the men who guided the CIA and NSA through the turn of the century to name the father of American cyberwar, and none hesitated: “Jim Gosler.” And yet in hacker circles Gosler remains an unknown.

Instead, he credits his colleagues and bosses in the intel world and a host of New Age management gurus. Gosler frequently cites Malcolm Gladwell—“The Outlier is fantastic!” he told me, more than once. Gordon Moore and Andy Grove, two former chief executives of Intel, were his heroes. Grove’s book Only the Paranoid Survive is his bible. But his all-time favorite is Price Pritchett, the organizational management guru.

Gosler’s office at CIA headquarters in Langley, they were greeted with the following Pritchett quote on the wall: Organizations can’t stop the world from changing. The best they can do is adapt. The smart ones change before they have to. The lucky ones manage to scramble and adjust, when push comes to shove. The rest are losers, and they become history.

The same year Gosler allegedly admitted to his Sandia coworkers that he was aiding the NSA’s exploitation work on a covert channel, a Swiss national by the name of Hans Buehler was arrested in Tehran for espionage.

Crypto AG, the German wing of it anyway, paid Tehran $1 million to release Buehler, even though, as far as Buehler knew, there was no truth to Tehran’s allegations. Not until three years later would two reporters at the Baltimore Sun—Scott Shane, who would later join the Times, and Tom Bowman, who joined NPR as a Pentagon correspondent—break the story of why Tehran would have reason to be suspicious. For years—as far back as World War II—the NSA had, with the CIA and Crypto AG’s blessing, and perhaps a helping hand from Sandia’s premier exploitation specialists, been spiking Crypto AG’s encryption machines to make any messages that passed through them easily decipherable to American decoders and analysts.

The opportunities to sabotage the global supply chain were endless, Gosler told me. My mind also darted back to Times headquarters, to Sulzberger’s closet, to one of those two classified NSA documents Glenn Greenwald was so hesitant to give up—the one that laid out, in intelligence jargon, how the NSA was spiking the global supply chain. The document was a 2013 NSA intelligence budget request, outlining all the ways the agency was circumventing encryption on the web. The NSA called it the SIGINT Enabling Project, and the vast reach of the agency’s meddling and incursion into the world’s digital privacy was disguised in typical agency lingo: The SIGINT Enabling Project actively engages the US and foreign IT industries to covertly influence and/or overtly leverage their commercial products’ designs. These design changes make the systems in question exploitable through SIGINT collection (e.g., Endpoint, MidPoint etc.) with foreknowledge of the modification. To the consumer and other adversaries, however, the systems’ security remains intact. In this way, the SIGINIT Enabling approach uses commercial technology and insight to manage the increasing cost and technical challenges of discovering and successfully exploiting systems of interest within the ever-more integrated and security-focused global communications environment.

We did not need degrees in computer science. We just needed to understand the relationship between foreign intelligence in digital form and human nature. We would exploit the relationship.

Five years earlier, the American intelligence community’s greatest fear was that the change in information flows would cause them to go blind or deaf. Now their greatest fear was drowning.

“Snowden was a low-level admin,” one former TAO hacker told me. “The NSA’s capabilities were far, far more expansive than what Snowden revealed.”

But nothing changed the surveillance game more than Apple’s unveiling of the first iPhone in 2007. TAO hackers developed ways to track an iPhone user’s every keystroke, text message, email, purchase, contact, calendar appointment, location, and search, and even capture live audio and video feeds of her life by hijacking her phone camera or hot-miking her microphone. The NSA swallowed up mobile alerts from travel companies—flight confirmations, delays, and cancellations—and cross-matched them with itineraries of other targets. An automated NSA program called Where’s My Node? sent analysts an email anytime an overseas target moved from one cell tower to another. TAO was now conducting instant, total invasion of privacy with minimal effort. And, until Snowden, iPhone users seemed blissfully oblivious to their invisible NSA ankle bracelet.

And as much as agency officials would later cite the NSA’s oversight by the Foreign Intelligence Surveillance Court, which by law had to approve any surveillance operation that might target Americans, the court had become something of a rubber stamp. The NSA’s arguments were heard without any opposing counsel, and when the court—under public pressure from the Snowden revelations—finally released numbers, it showed that of the 1,789 applications it received to surveil Americans in 2012, it approved 1,748 without any changes. Only one case had to be withdrawn.

By 2008 the NSA feverishly began removing human decision-making—and with it any complicated moral calculus—from their work. A highly classified NSA software program code-named Genie began aggressively embedding implants not just in foreign adversaries’ systems but in nearly every major make and model of internet router, switch, firewall, encryption device, and computer on the market. By 2013 Genie was managing 85,000 implants—four times the number of implants it had managed five years earlier—according to U.S. intelligence budgets, with plans to push that number into the millions. While three-quarters of those implants still prioritized targets in Iran, Russia, China, and North Korea, TAO had become far less discriminate.

American officials routinely point out that Huawei’s founder, Ren Zhengfei, “China’s Steve Jobs,” was a former Chinese PLA officer, and warn that Huawei’s equipment is riddled with Chinese backdoors. Chinese intelligence could use that access to intercept high-level communications, vacuum up intelligence, wage cyberwar, or shut down critical services in times of national emergency. That all may very well have been true. But it is also certainly true in reverse. Even as American officials were publicly accusing China of embedding trapdoors in Huawei’s products, my Times colleague David Sanger and I learned from leaked classified documents that the NSA had pried its way into Huawei’s headquarters in Shenzhen, years ago, stolen its source code, and planted its own backdoors in the company’s routers, switches, and smartphones.

No one asked what this might one day mean for the American tech companies they were breaking into, who were now servicing more customers abroad than in the United States. During the Cold War, the NSA did not have to reckon with this dilemma: Americans spied on Russian technology, while Russians backdoored American typewriters. But that was no longer the case. The world was now using the same Microsoft operating systems, Oracle databases, Gmail, iPhones, and microprocessors to power our daily lives. Increasingly, NSA’s work was riddled with conflicts of interest and moral hazards. Nobody seemed to be asking what all this breaking and entering and digital exploitation might mean for the NSA’s sponsors—American taxpayers—who now relied on NSA-compromised technology not only for communication but for banking, commerce, transportation, and health care. And nobody apparently stopped to ask whether in their zeal to poke a hole and implant themselves in the world’s digital systems, they were rendering America’s critical infrastructure—hospitals, cities, transportation, agriculture, manufacturing, oil and gas, defense; in short, everything that undergirds our modern lives—vulnerable to foreign attacks. There are no patents on vulnerabilities, exploits, and malware. If NSA found a way to exploit a digital system, there was a good chance that one day, months or years down the road, other bad actors would discover and exploit those very same weaknesses.

The Americans and Israelis then needed to design a payload, the actual instructions that would spin the rotors and destabilize the centrifuges. And they would need a way to convince Natanz’s technicians that all was well, even as their centrifuges spun into oblivion. There could be no fingerprints. No accidental misfires. And no impulsive strikes. The code would need to lie dormant, undetected, over time, so as not to blow their cover. Meeting even one of these needs was a feat. Meeting them all in tandem, over months and years, under cover of darkness, was an espionage coup of such unprecedented digital magnitude that it would later draw comparisons to the Manhattan Project.

We still don’t know exactly who brought the worm in. Some suspect a Mossad spy, a CIA officer, a Dutch mole, a well-paid insider, or an unwitting contractor at one of the five Iranian companies that Olympic Games targeted in the lead-up to the first attack. And we may only find out in 2039, when Olympic Games is set to be declassified. For now, all we know is that it had to have been a human with an infected thumb drive.

By 2013, Turbine was fully operational and began buffering TAO analysts from their operations. The robot was designed, in the words of one internal NSA memo, to “relieve the user from needing to know/care about the details.” By the end of the year, the NSA anticipated that Turbine would manage “millions of implants” for intelligence gathering and “active attack.” So fixated was the NSA on its new offensive cyber tools that year that offense trumped defense at the agency by a factor of two. The agency’s breaking-and-entering budget had swelled to $652 million, twice what it budgeted to defend the government’s networks from foreign attack. Its critics began to argue that NSA had abandoned its defensive mission entirely.

The NSA faced a quandary: its solution to dealing with the bad actors in the world was escalating an arms race that only made the United States more vulnerable to attack. The NSA’s answer to this problem was a system called Nobody But Us (NOBUS). The premise behind NOBUS was that low-hanging fruit—vulnerabilities that could easily be discovered and abused by American adversaries—should be fixed and turned over to vendors for patching. But more advanced exploitation—the kind of advanced zero-days the agency believed only it had the power, resources, and skills to exploit—would remain in the agency’s stockpile and be used to spy on American enemies or degrade their systems in the case of a cyberwar. Michael Hayden, the former NSA chief who ran the agency until 2005, addressed NOBUS this way: “You look at a vulnerability through a different lens if even with the vulnerability it requires substantial computational power or substantial other attributes, and you have to make the judgment: Who else can do this? If there’s a vulnerability here that weakens encryption, but you still need four acres of [supercomputers] in the basement in order to work it, you kind of think NOBUS, and that’s a vulnerability we are not ethically or legally compelled to try to patch—it’s one that ethically and legally we could try to exploit in order to keep Americans safe from others.”

The zero-days Desautels brokered had to be in an “ideal state”—that is, they required zero interaction on the target’s end: no spammy text messages or phishing emails like the ones Chinese hackers were known to send. The exploits Desautels developed and brokered had to work 98.9 percent of the time. And if they failed, then they had to “clean fail”—meaning they couldn’t trigger a security alert or crash a target’s computer. No one could know they were being hacked. The operations were simply too sensitive. If the target caught even the faintest whiff they were being targeted, it was game over.

They were after Google’s Gmail accounts and its source code. Most laypeople assume hackers are after short-term payoffs: money, credit card information, or bribe-worthy medical information. But the most sophisticated attackers want the source code, the hieroglyphics created and admired by the engineering class. Source code is the raw matter for software and hardware. It is what tells your devices and apps how to behave, when to turn on, when to sleep, who to let in, who to keep out. Source code manipulation is the long game.

Back in 2011, a whistleblower tipped off the Pentagon that its security software was riddled with Russian backdoors. The Pentagon had paid Computer Sciences Corporation—the same megacontractor that now owns VRL—$613 million to secure its systems. CSC, in turn, subcontracted the actual coding to a Massachusetts outfit called NetCracker Technology, which farmed it out to programmers in Moscow. Why? Greed. The Russians were willing to work for a third of the cost that U.S. programmers had quoted. As a result, the Pentagon’s security software was basically a Russian Trojan horse, inviting in the very adversary the Pentagon had paid hundreds of millions of dollars to keep out.

All optimism evaporated in 2014, when the Russians took their attacks one step further. That January, CrowdStrike discovered that Russian hackers had successfully compromised industrial control software companies and Trojanized the software updates that made their way into hundreds of industrial control systems across the country. It was the same technique the Americans and Israelis had used five years earlier with Flame, when they infected computers in Iran using Trojanized Microsoft software updates.

This was not Chinese-style industrial espionage. Moscow was preparing the battlefield. “This was the first stage in long-term preparation for an attack,” John Hultquist, a top threat researcher, told me. “There’s no other plausible explanation. Let’s just say they’re not there to collect intelligence on the price of gas.”

Included in Sandworm’s files was a telling clue. Sandworm wasn’t after emails and Word docs. It was targeting files used by industrial engineers. One of Trend Micro’s researchers had previously worked at Peabody Energy, the world’s largest coal producer. This gave him a unique window into what they were seeing. Sandworm’s attackers were targeting “.cim” and “.bcl” files, two file types used by General Electric’s industrial control Cimplicity software—the same software Peabody’s engineers used to remotely check on their mining equipment. That very same GE software was used by industrial engineers the world over. It was a human-machine interface used to check on the PLCs that control the world’s water treatments facilities, electric utilities, transportation companies, and oil and gas pipelines. Trend Micro’s researchers peeled the code back further and discovered that it was designed to install itself, execute code, then immediately delete itself once the job was done. Among its various commands were the words “die” and “turnoff ”—the first step in sabotaging the machinery on the other end. Russia’s hackers weren’t in those systems to play; they were looking to exact harm.

It was an NSA computer algorithm that had come up with the name, but Eternal ended up being a fitting moniker for a set of zero-day exploits that would haunt Daniel, the NSA, and American businesses, towns, and cities for years to come. One of those exploits, EternalBlue, targeted critical bugs in a Microsoft software protocol called the server message block (SMB). The protocol enabled computers to pass information, like files or printer services, from server to server at internet speed.

It took a team of some of the NSA’s best analysts to develop the algorithm that ensured that EternalBlue would land on the target’s computer without crashing screens on the other end. And once they figured it out, TAO marveled at the magic of their polished espionage tool. “It netted some of the very best counterterrorism intelligence we got,” one former TAO hacker told me. One of EternalBlue’s best attributes was that it wasn’t “dirty”—it left minimal logging behind. It allowed the agency’s hackers to move from server to server undetected. The chance that the NSA’s targets—terrorists, Russia, China, North Korea—would ever discover they’d been had with the agency’s exploit was virtually null. The agency used EternalBlue for espionage. But they knew if the exploit ever got out, it could just as easily function as an intercontinental missile. If hackers in Iran, North Korea, China, Russia, or God knows where else swapped out the payload for one that could sabotage data or shut down systems on the other side, it could wreak total havoc.

Inside the CIA, officials knew with 100 percent certainty that this was the Kremlin’s dirty work, but their intelligence relied on highly sensitive American spies inside Putin’s network. The agency worried that making anything public would put its CIA sources in danger. And Obama worried that by definitively declaring the hacking campaign a Russian operation, he would be seen as interfering in the election too.

A century and a half earlier, in 1949, he reminded the crowd, a dozen countries had come together to agree on basic rules of warfare. Hospitals and medical personnel were off limits, the countries agreed. It took three more diplomatic summits over the next century before 169 nation-states signed on to the Fourth Geneva Convention, agreeing to basic protections for wounded or captured military personnel, medical personnel, and nonmilitary civilians during wartime—rules that still hold today. “It was here in Geneva in 1949 that the world’s governments came together and pledged that they would protect civilians even in times of war,” Smith told government officials from around the globe. “And yet let’s look at what is happening. We’re seeing nations attack civilians even in times of peace.”

Under Trump, things unraveled much more quickly, in a dimension few Americans could truly grasp. The agreement Obama had reached with Xi Jinping to cease industrial espionage ended the day Trump kicked off his trade war with China. Trump’s abandonment of the Iran nuclear deal—the only thing keeping Iran’s hackers on good behavior—unleashed more Iranian cyberattacks on American interests than ever before. The Kremlin—which had yet to feel much of any pain for its 2016 election interference or its hacks on the Ukraine and U.S. grids—never stopped hacking our election systems, our discourse or our infrastructure. Our fickle allies in the Gulf—the Saudis and Emiratis—only became more brazen in their choice of targets. Having avoided so much as a slap on the wrist for their brutal murder of Saudi journalist Jamal Khashoggi, the Saudis blinked and kept going.

Three years after the NSA lost control of its tools, the long tail of EternalBlue was everywhere. The underlying Microsoft bugs were no longer zero-days—a Microsoft patch had been available for two years—and yet EternalBlue had become a permanent feature in cyberattacks on American towns, cities, and universities, where local IT administrators oversee tangled, cross-woven networks made up of older, expired software that stopped getting patched long ago. Not a day went by in 2019, Microsoft’s security engineers told me, when they did not encounter the NSA’s cyberweapons in a new attack.

Months before the Shadow Brokers first leaked the NSA’s tools in 2016—and more than one year before North Korea and Russia used them to wreak global havoc—China had discovered the NSA’s exploits on their own systems, snatched them, and used them for their own stealth attacks. It took three years for anyone to sort this out.

Symantec’s discovery was clear evidence that even when the NSA used its tools in stealth, there were no guarantees that our adversaries wouldn’t detect them

More unsettling was the Chinese hacking group behind the extraction and redeployment of the NSA’s exploits. The group, code-named Legion Amber, was based in Guangzhou, an ancient city in southern China, but even the agency struggled to make sense of their ties to the state. Legion Amber’s “operators appear to be private or contract hackers, but little is known about their affiliation,” one classified NSA assessment concluded. “However, their heavy targeting of Five Eyes and global government and industrial entities suggests they are operating on behalf of elements of the Chinese government.” NSA analysts came to believe that Legion Amber’s members were part of a digital reserve army comprising China’s top security engineers, who worked for private internet companies by day but were tapped by China’s main spy agency, the Ministry of State Security, for sensitive operations by night.

China was decades behind the United States in nuclear weapons development, but thanks to Legion Amber, it had stolen everything it needed to catch up. In 2018, U.S. officials watched in horror as Beijing successfully tested a new submarine-launched ballistic missile and began moving ahead with a new class of subs that could be equipped with nuclear-armed missiles.

In early 2019, I discovered that Boeing, General Electric Aviation, and T-Mobile had all been targeted. Within a year, China’s hit list had expanded to include as many telecom, manufacturing, health care, oil and gas, pharma, high tech, transportation, construction, petrochemical, travel, and utilities companies and universities as they could break into. Only this time, rather than brute-forcing their way into victims directly, China’s hackers were coming in through side doors, breaking into companies via the software employees use to work remotely. They’d abandoned malware commonly attributed to China and started encrypting their traffic. They cleaned up after themselves, erasing server logs and moving files to Dropbox rather than sucking them directly back to China’s command-and-control servers.

Skeptics argue that Xi never planned to stick to the 2015 agreement in the first place. Former Obama officials maintain that Xi was sincere, that the deal would have stuck had Trump not flipped the tables over. What we do know is that in the three years since Xi signed the deal, he had consolidated PLA hacking divisions under a new Strategic Support Force, similar to the Pentagon’s own Cyber Command, and moved much of the country’s hacking operations away from the PLA’s scattershot hacking units to the stealthier and more strategic Ministry of State Security.

Beijing started hoarding its own zero-days, eliminating any above- or belowground market for them in China. Authorities abruptly shuttered China’s best-known private platform for reporting zero-days and arrested its founder.

As Ukraine was to Russia, Xinjiang was now to China—an incubator for every new piece of surveillance technology. Uighurs were forced to download compulsory spyware that monitored their calls and messages. Surveillance cameras now hung from every doorway, shop, mosque, and street in Xinjiang. Facial recognition algorithms had been trained to identify Uighurs by their unique facial features. And when they snagged one, China’s minders inspected every pixel of the footage, sniffing for any whiff of dissent.

“The Chinese use their best tools against their own people first because that’s who they’re most afraid of,” Jim Lewis, the former government official who tracked cyber threats, told me. “Then they turn those tools on us.”

This, you see, was the precarious state of the Iranian cyber threat when—on January 2, 2020—Trump ordered up a drone strike on General Qassim Suleimani. The United States could have killed Suleimani a thousand times before. But previous administrations had never dared pull the trigger for fear his death would provoke the kind of large-scale retaliation that would lead to war. Iran’s powerful security and intelligence commander was like a second son to Iran’s Supreme Leader, Ali Khamenei. He led the Revolutionary Guards’ Quds Force, and was responsible for the killing of hundreds, if not thousands, of Americans in Iraq over the years. No doubt he had more planned. But inside Iran, he was a hero. “Buckle your seatbelt,” one senior official texted me the night Suleimani was blown to smithereens.

More recently, intelligence analysts determined that a prominent Russian cybercriminal—the leader of an elite cybercrime group that called itself Evil Corp—wasn’t just working hand-in-hand with the FSB. He was FSB. “There’s a pax mafiosa between the Russian regime and its cyber cartels,” is how Tom Kellermann, a Russian cybercrime expert, put it to me as we inched closer to the 2020 election. “Russia’s cybercriminals are treated as a national asset who provide the regime free access to victims of ransomware and financial crime. And in exchange, they get untouchable status. It’s a protection racket and it works both ways.”

“The mantra of Russian active measures is this: ‘Win through force of politics rather than the politics of force,’ ” is how Clint Watts, a former FBI agent who specializes in Russian disinformation, explained it to me. “What that means is go into your adversary and tie them up in politics to the point where they are in such disarray that you are free to do what you will.”

Starting in late September, U.S. Cyber Command started hacking into TrickBot’s command and control, feeding its infected computers a set of instructions that sent them into an endless loop. It was the equivalent of a phone dialing its own number over and over and over again so that anyone else trying to get through would get a busy signal. TrickBot’s operators were able to reclaim their computers in half a day. But roughly a week later, Cyber Command struck again, hitting TrickBot’s systems with the same attack as before. This time too the disruption was temporary. But it was also a message, similar to the one General Nakasone’s cyber troops sent the Internet Research Agency in the days leading up to the midterms: We are watching you, we are inside, and we will take you out if you come for our election. (As the election approached, we learned that Cyber Command launched a similar attack on the Iranian hackers behind the Proud Boys campaign, hindering their ability to conduct more attacks.)

I would like to think it was the coordinated attacks by Cyber Command, the unsung heroes at CISA who zipped up state and county systems, the TrickBot takedowns, the quick attribution of the Iranian attacks, or the naming and shaming by federal prosecutors who, in the weeks leading up to the election, unsealed charges against the Russian military intelligence officers behind NotPetya, the Ukraine grid attacks, the attack on the 2018 Olympics, the French election, and the probes of our voter registration databases in 2016. I would like to think that collectively all of it amounted to successful deterrence that we could improve upon and deploy again and again.