Zucked: Waking Up to the Facebook Catastrophe

by Roger McNamee

The AG’s office had the skills and experience to handle the most complex cases. In time, we would furnish them with whistle-blowers, as well as insights. By April 2018, thirty-seven state attorneys general had begun investigations of Facebook.

Technological progress has merely provided us with more efficient means for going backwards. —ALDOUS HUXLEY

we got a call from Washington asking that we return. They invited us for three days, and they scheduled every moment. Just in time for the trip, Renée DiResta joined our team. Renée is one of the world’s foremost experts on how conspiracy theories spread

over the internet. In her

day job, as director of research at New Knowledge, Renée helps companies protect themselves from disinformation, character assassination, and smear attacks, the kind of tactics the Russians used in 2016. New Knowledge also created Hamilton 68, the public dashboard that tracks Russian disinformation on Twitter. Sponsored by the German Marshall Fund and introduced on August 2, 2017,...

Renée is also director of policy of Data for Democracy, w...

be an inclusive community of data scientists and technologists to volunteer and collaborate on projects that make a positive impact on society.” Renée’s own focus is on analysis of efforts by bad actors to subvert democracy around the world. Unlike us, Renée was a pro in the world of election security. She and her colleagues had heard whispers of Russian interference efforts in 2015 but had struggled to get the authorities to take action. The daughter of a resear...

Yonkers, New York, Ren...

love was music. She learned to play piano as a child and played it “competitively” into her college years. Renée started coding at age nine and volunteered in a research lab at the Sloan-Kettering hospital in eighth grade. She worked on a project that looked for a correlation between music training and temporal reasoning. It was a small-scale study, but it married Renée’s two great interests. She earned a degree in computer science at SUNY Stony Brook before going into government service in a technology operations role. Renée underplays this part of her résumé, focusing instead on her time in algorithmic trading on Wall Street. In that jo...

When a Group member shares a news story, the other members will generally give it the

benefit of the doubt. When Group members begin to embrace that story, the pressure mounts on other members to do the same. Now imagine a Facebook Group that includes a Russian troll and enough Russian-controlled bots to represent 1 to 2 percent of the membership. In that scenario, the trust

of Group members leaves them particularly vulnerable to manipulation. A story like Pizzagate might get its start with the troll and bots, who would share it. Recipients assume the story has been vetted by members of a shared political fr...

conspiracy theories can gain traction quickly. The Russians used this technique to interfere in the 2016 election. They also organized events. A particularly well-known example occurred in Houston, Texas, where Russian agents orga...

at the same mosqu...

same time. The goal was to trigger a ...

All of this is possible because users trust what they find on social media. They trust it because it appears to originate from friends and, thanks to filter bubbles, conforms to each user’s preexisting beliefs. Each user has his or her own Truman Show, tailored to press emotional buttons, including those associated with fear and anger. While endless confirmation of their preexisting beliefs sounds appealing, it u...

democracy and government, and, ultimately, to favor one candidate...

Adding Renée to our team was transformational. Unlike Tristan and me, Renée could go beyond hypotheses. She had been researching this stuff for several years. She lived in the world of facts. Every member of Congress and

staffer we met took to her immediately. What Renée got from us was a new platform to share what she knew. She had been working at Data for Democracy, identifying threats early and asking the tech companies to take the spread of computational propaganda more seriously. As so often happens with researchers and intelligence professionals, their brilliant insights did not always reach the right people at the right time.

They understood that an algorithm is, as Wikipedia

defines it, “an unambiguous specification of how to solve a class of problems,” generally related to calculation, data processing, and automated reasoning. But with advances in AI, algorithms have become more complex as they adapt, or “learn,” based on new data. In Facebook’s relentless effort to eliminate any friction that might limit growth, they automate everything, relying on ever-evolving algorithms to operate a site with 2.2 billion active users and millions of advertisers. Algorithms find patterns shared by different users, based on their online behavior. This goes way beyond their common interests to include things like the time, location, and other context elements of web activity. If User A does a dozen things online prior to buying a new car—many of them unrelated to buying a car—the algorithm will look for other users who start down the same path and then offer them ads to

buy a car. Given the complexity of Facebook, the AI requires many algorithms, the interaction of which can sometimes produce unexpected or undesirable outcomes. Even the smallest changes to one algorithm can trigger profound ripple effects through the rest of the system. A clear case where moving fast can break things in unpredictable ways. While Facebook argues that its technology is “value neutral,” the evidence suggests the opposite. Technolo...

algorithms as correlating data from individual users and between users. In an opinion piece in The Guardian, Lanier wrote, “The correlations are effectively theories about the nature of each person, and those theories are constantly measured and rated for how predictive they are. Like all well-managed theories, they improve through adaptive feedback.” When it comes to the algorithms used by internet platforms, “improve” refers to the goals of the platform, not th...

mean they are fair. When used to analyze mortgage applications, for example, algorithms that reflect the racial biases of their creators can and do harm innocent people. If the creators of algorithms are conscious of their biases and protect against them, algorithms can be fair. If, as is the case at Facebook, the creators insist that technology is by...

The CEOs of any other industry would have been there. But that did not happen with Facebook, Google, and Twitter. For reasons that underscore the partisan divide in Congress, the majority did not insist on testimony from the CEOs. They settled for the general counsel of each company. These are very well-educated, successful lawyers, but they are not engineers. They are good at talking, but their familiarity with the inner workings of their firm’s products would have been limited. If the goal was to minimize the effectiveness of the hearings from the

outset, they were the

perfect wit...

They knew what they did not know and were not

afraid to admit it.

The Russians did not reach a random set of 126 million people on Facebook. Their efforts

were highly

targeted....

Russians ran a number of Facebook Groups targeting people of color, including Blacktivist, which gained a substantial following in the months before the election. They ran another Group called United Muslims of America, with a similar approach to a different audience. On Twitter, the Russians

ran accounts like “staywoke88,” “BlackNewsOutlet,” “Muslimericans,” and “BLMSoldier,” all designed, like Blacktivist and United Muslims of America, to create the illusion that genuine activists supported whatever positions the Russians promoted. In an election where only 137 million people voted, a campaign that targeted 126 million eligible voters almost certainly had an impact. How would Facebook spin that?

Stretch held his own until relatively late in the hearing, when Senator John Kennedy, a Republican from Louisiana, surprised the whole world—and especially some of his Republican colleagues—by posing a series of questions to Stretch about whether Facebook had the ability to look at personal data for individual users. Stretch tried to deflect the question by answering that Facebook had policies against looking at personal

data. Hidden behind an

“aw shucks, I’m a country lawyer” demeanor, Kennedy has a brilliant mind, and he reframed his question until Stretch was forced to answer yes or no. Did Facebook have the ability to look at a user’s personal data? Kennedy reminde...

at Facebook, rules often are more about preventing liability than about changing behavior—but

In the end, Stretch and

the other general counsels managed to dodge the toughest questions posed by the Senate Judiciary Committee, eliciting a rebuke from the ranking member, Senator Dianne Feinstein.

The Republican members, under the leadership of committee chair Devin Nunes, were holding

one kind of hearing, while the Democrats, led by ranking member Adam Schiff, held another.

Televised congressional hearings are typically long on theater and short on substance. That is especially true in an environment as polarized as Capitol Hill in 2017. From our perspective, the normal rules did not apply to these hearings. By making millions of Americans aware that internet platforms had been exploited by the Russians to interfere in our presidential election, these

hearings had done something important. They were a first step toward congressional oversight of the internet platforms, and we had played a small role.

The Facebook Way

The problem isn’t any

particular technology, but the use of technology to manipulate people, to concentrate power in a way that is so nuts and creepy that it becomes a threat to civilization. —JARON LANIER