The Imagineers of War: The Untold Story of DARPA, the Pentagon Agency That Changed the World

by Sharon Weinberger

U.S. Central Command headquarters, General Norman Schwarzkopf, who was directing the war, reviewed the simulations. “We were instantiating it in the simulation, saying, ‘This is a good idea but this one is not,’ and ‘Here’s why,’ and he would readjust,” McBride recalled. “He personally planned that very first mission with us, with the simulation capability.”

few hours later, an air force F-117 Nighthawk—a stealth aircraft derived from DARPA’s Have Blue prototype—flew safely through that air corridor. The F-117’s first bomb destroyed an Iraqi air force site, and the second leveled a telecommunications hub in the center of Baghdad.

the first Joint STARS aircraft, a DARPA-sponsored airborne tracking radar, spotted a massive convoy of Iraqi vehicles fleeing Kuwait and passed the data directly to strike aircraft. The resulting destruction of some two thousand Iraqi vehicles earned the escape route the name “highway of death,” and Air Force Magazine praised DARPA’s airborne radar as “one of the more unlikely heroes of Operation Desert Storm.”

Yet in the Pentagon, officials were not interested in any ambitious new DARPA projects.

Thorpe had a reputation for bending the rules. When DARPA was told its simulator building at Fort Knox, Kentucky, was possibly in violation of the law because all new military construction required prior congressional authorization, he had a trailer hitch installed and claimed it was just a temporary structure.

Now Thorpe had an idea that seemed to match the “cost cutting” mission for DARPA. He said there had been a big tank battle during the Gulf War, just a few days prior. Thorpe wanted to re-create the battle in a virtual world of computers, something that had never been done before and could potentially save money on training. The key, however, was that he wanted to send scientists to a battleground in Iraq strewn with still smoldering Iraqi tanks.

“Sure,” the director told him. And with that, DARPA set off on one of its most ambitious post–Cold War projects: to re-create real war in a virtual world, based on data gathered from the smoking ruins of a battlefield.

Normally, after major battles, army historians would be sent to interview participants and chronicle events as part of a written account of what took place. Thorpe wanted DARPA to send simulation experts to the battlefield, walk among the burned-out tanks, interview American soldiers who had fought there, and then plug that data into the virtual reality world of SIMNET. The entire battle could be re-created and played in a simulator, and more important, those simulators could be connected on a network, sending data packets back and forth, so that people could replay the battle as participants. “It would be like a living history,” Thorpe suggested.

Everyone loved it, and Cheney most of all. “Gee, if we had this earlier, I could have shown this to Hussein, and he might have realized in how bad shape he was and just given up,” Cheney said, perhaps foreshadowing misplaced optimism he would have more than a decade later about Iraq.

Yet it is less clear that SIMNET, and by extension 73 Easting, really had much practical effect on the military. The reality is that SIMNET was fielded too late to help the tank operators who fought at 73 Easting, though it was used in later years.

Using simulations to practice for the last conflict did not do anyone any good.

“We built this incredibly complex end-to-end model from seed planting down in South America through the changing to a product at a wholesale level, the transportation across myriad modes of transportation, ultimately into warehouses in the United States of America,” McBride said. Yet the more DARPA modeled the problem, the worse it looked. If one cartel was defeated, it ended up just strengthening another cartel. Like the Greek Hydra, if you cut off one head, two more rose in its place. DARPA came up with answers, but the answers did not fit what the White House wanted.

No matter which way DARPA modeled the drug war, it could not come up with a scenario that cut off the supply.

one thing DARPA’s simulations could never do well was to model human beings.

The DARPA director that year authorized the start of a new program, called Total Information Awareness. It sought nothing less than to predict human behavior, in particular the behavior of terrorists.

At DARPA, however, Goldblatt turned his attention to something far more ambitious than food wrappers: he wanted to work on enhancing human beings. Goldblatt was inspired by science fiction like Firefox, the 1982 movie starring Clint Eastwood, which featured weapons controlled by the human mind. Under Goldblatt, DARPA funded a group of researchers at Duke University to implant microelectrodes in monkeys’ brains.

An FBI supervisor in the Minneapolis field office, which had started the inquiry, got into an argument with headquarters about the importance of tracking Moussaoui. Pursuing the investigation was critical, the Minneapolis-based agent argued, “to keep someone from taking a plane and crashing into the World Trade Center.”

Watching the day’s news unfold, Tether was convinced that the problem had been not a lack of data but a failure to centralize and analyze the data.

The “laboratory” was all smoke and mirrors, at least in the fall of 2001. A Hollywood set designer had been hired to create the futuristic-looking command-and-control center with large, sleek display screens and flashing lights. The humming computers were not churning any real data. There was research going on at companies and universities, but the laboratory was just a showcase to convince intelligence officials that data, or more important, spotting patterns in data, could help predict the next terrorist attack.

More notable than the Hollywood-inspired set was the wizard behind the curtain: John Poindexter, a retired admiral who had once been Ronald Reagan’s national security adviser. A PhD physicist and technology enthusiast, Poindexter burnished his image in public memory as the relaxed, pipe-smoking witness during the 1987 Iran-contra hearings, which investigated the Reagan administration’s sale of arms to Iran. Poindexter helped orchestrate the convoluted deal, which was also used to finance the contras in Nicaragua, in violation of law. He then systematically shredded evidence of the scheme once it came to light.

Tether hired Poindexter to run an entirely new office in DARPA, called the Information Awareness Office, with plans to spend more than $200 million in its first two years, and a flagship project called Total Information Awareness.

The reason Tether thought he could do something as audacious as hire a figure at the heart of one of the greatest modern national political scandals, let alone put him in charge of a high-profile counterterrorism program, had to do with Chen...

His reputation for being computer savvy helped propel him to the White House in the early 1980s, where he was assigned to modernize the Crisis Management Center in the Old Executive Office Building. Equipped with fiber-optic cable, the revamped center introduced videoconferencing to the White House. Poindexter also brought PROFS Notes, an early version of e-mail, to the National Security Council staff.

Poindexter was introduced to Brian Sharkey, a program manager at DARPA who was interested in analyzing data to help predict political crises. Soon, Poindexter was working under contract to DARPA. Together, Sharkey and Poindexter in 1996 launched a DARPA-sponsored data-analysis program called Collaborative Crisis Understanding and Management, later changed to Genoa (because both were former naval officers, they liked the idea of naming the program after a sail).

Just days after the 9/11 attacks, Poindexter was sitting in front of DARPA’s new director with a briefing presentation titled “A Manhattan Project for Combating Terrorism.” Poindexter pitched his vision of a massive technology program to combat terrorism on the scale of the World War II race to build the atomic bomb. Poindexter’s idea was to create a massive data-mining system capable of aggregating databases across government and the private sector and then pulling out warnings of the next September 11 attack. Poindexter proposed another Manhattan Project, made up of top researchers from government, academia, and industry. Poindexter even half joked that he would put everyone “in a compound with barbed wire around it” so the people inside could not leave until they had solved the terrorism problem.

One of the slides Poindexter presented to Tether was particularly striking: it laid out a $100 million unclassified “white” program, called Total Information Awareness, and then a parallel secret “black” program that would have five times that budget. Operating in strict secrecy, this highly classified black program would be called Manhattan Project Terrorism.

Tether encouraged Poindexter to go to credit card companies and collect commercial data, but Poindexter said he balked at the idea of using real-world data for a research program, knowing that could immediately raise public objection. Even though the idea was eventually to create a centralized database of real information, Poindexter decided, for the time being, to use made-up data.

The pyramid, though a familiar image, is also a symbol often linked to conspiracy theories. Yet no one at DARPA seemed to think that was a problem.

“I know six good ways to smuggle a nuclear weapon into the United States,” Lukasik told Tether during a private meeting in the director’s office.

In June 1975, more than twenty-five years before the Total Information Awareness imbroglio, a series of sensational news reports warned of a new Orwellian-sounding computer technology that would be used to create dossiers on individual Americans. “What this technology means to you is this: The Federal Government now has the means to put together a computer file on you, or almost any American, within a matter of minutes,” Ford Rowan, a correspondent for NBC Nightly News, reported. “The key breakthrough in the new computer technology was made by a little known unit of the Defense Department—the Advanced Research Projects Agency, ARPA.” The technology involved was the ARPANET.

news reports claimed that the government was using the ARPANET to create centralized dossiers through a secret network that linked the White House, the CIA, the Defense Department, the FBI, and the Treasury Department. None of that was true.

One of the ISAT members, Eric Horvitz, had already been thinking about how to use computers to help sift through large amounts of data to predict future events. But that year, Horvitz, a prominent artificial intelligence expert working at Microsoft, saw an opportunity to apply this work to the nexus of data mining and privacy. His vision was for something called selective revelation.

He even attended the summer study meeting in 2002 held at the offices of the Institute for Defense Analyses in Alexandria, Virginia. ISAT invited two privacy advocates to observe the meeting, including Marc Rotenberg, the head of the Electronic Privacy Information Center. Poindexter and Rotenberg had clashed previously in the 1980s, when Poindexter was in the White House and Rotenberg worked as a congressional staffer.

According to Poindexter, Rotenberg said he understood, adding that he believed there needed to be more oversight. Poindexter took that as an encouraging sign. He was wrong, however. Rotenberg saw the meeting from a completely different perspective. In his view, Poindexter and others involved in the project had no understanding of what privacy meant.

The Information Awareness Office had as its head the bête noire of Iran-contra, an office seal that featured an icon of Illuminati-inspired conspiracy theories, and an ambitious vision for a centralized database. The new office was ready for its public debut, and for that it headed to Disneyland.

DARPATech came and went, generating only modest press coverage, mostly focused on Tether’s announcement that DARPA would sponsor a “robot race” in the California desert, a demonstration of self-driving cars.

The next week, the New York Times columnist William Safire declared war on Total Information Awareness, calling it an affront to the American way of life.

Safire’s column was a conflation of fact and fantasy: Total Information Awareness was a research project using make-believe data, and there were plenty of legal reasons why some of that data might be excluded even assuming the technology was eventually adopted. On the other hand, Safire’s description was a reasonable portrayal of the scope of Poindexter’s ambition.

Total Information Awareness was, in his view, just a research program. Yet the articles were treating the project as if it were an operational system collecting everybody’s medical records.

When congressional staffers combed through the report, one project in particular caught their attention: a small research study for something called the FutureMAP, which looked at the potential of using the “wisdom of crowds,” represented by free-market investors, to predict future political events. FutureMAP actually had its origins with a researcher from the National Science Foundation, who was interested in the predictive capabilities of open markets. DARPA had awarded contracts for very preliminary work, which would fund researchers to test whether having people wager real money on future political events would make for accurate predictions.

one of the companies involved, created a website that featured “colorful examples,” like “the assassination of Arafat, and a missile attack from North Korea.”

Poindexter’s resignation letter to Tether in August 2003 ran five single-spaced pages, expounding on DARPA’s origins, its purpose, and its achievements. He reviewed the history of the Information Awareness Office, providing a detailed and unapologetic defense of its activities, and blasted the “charged political environment of Washington, where glib phrases, ‘sound bites,’ and symbols” are used instead of debate. He expressed hope that Congress might salvage some of his office’s work.

Total Information Awareness never died; it just went black, just as Poindexter had proposed at the start. The Advanced Research and Development Activity, part of the NSA, took over almost all the programs that had been in the Information Awareness Office, except the privacy protection research. “The critics we had got the worst of all worlds,” Poindexter pointed out.

The controversy not only ended the related privacy research; it also pushed data mining deep into the classified world of the intelligence community, laying the intellectual groundwork for the massive analysis and collection system that would be revealed ten years later by an NSA contractor named Edward Snowden.

The DARPA director soon faced his own crisis. Shortly after the controversy, Newt Gingrich, the former Speaker of the House, paid him a visit with a direct message from Rumsfeld, who rarely communicated directly with DARPA’s director: “I came over to tell you that the Chief says you’re getting close [to getting fired].”

“Writers such as H. G. Wells, who for example wrote in his 1914 novel The World Set Free about nuclear power and talked about the atomic bomb and gave it the name used today, would have been a great DARPA [program manager].”

“Imagine 25 years from now, where old guys like me put on a pair of glasses or a helmet and open our eyes,” Tether said in a DARPATech speech, referring to the work in Goldblatt’s office. “Somewhere there will be a robot that will open its eyes, and we will be able to see what the robot sees. We will be able to remotely look down on a cave and think to ourselves, ‘Let’s go down there and kick some butt.’

More than anything, Tether loved Disneyland. The man who would become the agency’s longest-serving director believed the home of Mickey Mouse and Walt Disney’s Enchanted Tiki Room represented everything he envisioned for DARPA.

He also embraced novelties like “polymer ice,” a synthetic substance that could be thrown from the back of Humvees to make an enemy slip off the road.

He supported work, for example, on a controversial “hafnium bomb,” which used a radioactive material that would potentially be tens of thousands of times more powerful per gram than conventional explosives, if scientists could figure out a way to trigger it. None could.