More on this book
Community
Kindle Notes & Highlights
Read between
May 4 - June 17, 2024
we have fallen for the idea that these services are “free.” In reality, we pay with our data into a business model of extracting human attention.
CA shifted from instigating tribal conflict in Africa to instigating tribal conflict in America.
we assume there’s some guy with a secret team of experts sitting in an office with a plan, and if that plan doesn’t work, don’t worry, he’s got a plan B and a plan C—someone in charge will take care of it. But in truth, that guy doesn’t exist. If we choose to wait, nobody will come.
Computer models are not magical incantations that can predict the world—they can make predictions only when there is an ample amount of data to base a prediction upon.
Exploring the nuances of identity and personality started to help me unpack why, despite the fact that politicians do polling all the time, they still seem horrendously out of touch. This is because so many of their pollsters are out of touch. Polling firms influence politicians’ ideas of what makes up voter identity, which are usually horrendously oversimplified or just plain wrong.
Extremism starts with how people look and how society feels. Sometimes it creates literal uniforms: Olive tunics and caps with red stars, red armbands, white pointed hoods, polo shirts and tiki torches, MAGA hats. These uniforms, in turn, are incorporated into the wearer’s identity, transforming their thinking from This is what I believe into This is who I am.
Even terrorists have pop culture.
People high on the narcissism scale are susceptible because they are more prone to feelings of envy and entitlement, which are strong motivators of rule-breaking and hierarchy-defying behavior. This means these targets will be more likely to develop an exaggerated suspicion of harassment, persecution, victimhood, or unfair treatment. This is the “low-hanging fruit” for initiating the subversion of a larger organization. Later, this learning would serve as one of the foundations for Cambridge Analytica’s work catalyzing an alt-right insurgency in America.
What the social data offered was a trail of detailed personal information that previously would have taken months of careful observation to gather. The targets were in effect creating their own dossiers with rich data that could quicken a psychologist’s assessment of their disposition.
Gamergate was not instigated by Breitbart, but it was a sign to Bannon, who saw that angry, lonely white men could become incredibly mobilized when they felt that their way of life was threatened. Bannon realized the power of cultivating the misogyny of horny virgins. Their nihilistic anger and talks of “beta uprisings” simmered in the recesses of the Internet.
Some biases are so common and seemingly intuitive that it can be hard for people to even recognize that they are actually irrational.
He reads about intersectional feminism or the fluidity of identity not, as I later learned, because he’s open to those ideas but because he wants to invert them—to identify what people attach themselves to and then to weaponize it. What I didn’t know that day was that Bannon wanted to fight a cultural war,
It turns out that Republicans can accept a batshit insane candidate, so long as it’s consistent insanity. This finding later informed almost everything that Cambridge Analytica worked on.
Even some state governments will sell you lists of hunting, fishing, or gun licensees. Did the state government bureaus care, or even bother to ask, where this data on their citizens was going? Nope.
People would feel better about their day after an hour-long session in the Fox News rage room—they could groan out their stress, and afterward their problems at work or home were someone else’s fault. It meant that their struggles could be wholly externalized, sparing them the stark reality that maybe their employer didn’t care enough about them to give them a living wage. It would be too painful to admit that perhaps they were being taken advantage of by someone they saw every day rather than the faceless enemy of Obamacare and “illegals.”
anger disrupts the ability to seek, rationalize, and weigh information. This leads to a psychological bias called affect heuristic, where people use mental shortcuts that are significantly influenced by emotion.
When it comes to what’s happening in other places, Americans will talk about “tribes,” “regimes,” “radicalization,” “religious extremists,” “ethnic conflicts,” “local superstitions,” or “rituals.” Anthropology is for other people, not Americans. America is supposedly this “shining city upon a hill,” a term Ronald Reagan famously popularized, adapting it from the biblical story of the Sermon on the Mount. But when I would see evangelists prophesying the end times and woe unto the nonbelievers, when I watched a Westboro Baptist Church demonstration, when I saw a gun show with bikini-clad ladies
...more
friends, colleagues, spouses, and parents typically see only part of your life, where your behavior is moderated by the context of that relationship.
Some of the staff working at Palantir realized that Facebook had the potential to become the best discreet surveillance tool imaginable for the NSA—that
As corny as this might sound, it really felt like I was working on something important—not just for Mercer or the company, but for science. However, I let this feeling distract me to the point of allowing myself to excuse the inexcusable.
Cambridge Analytica would target them, introducing narratives via Facebook groups, ads, or articles that the firm knew from internal testing were likely to inflame the very narrow segments of people with these traits. CA wanted to provoke people, to get them to engage.
When someone follows pages of generic brands like Walmart or some prime-time sitcom, nothing much changes in his newsfeed. But liking an extreme group, such as the Proud Boys or the Incel Liberation Army, marks the user as distinct from others in such a way that a recommendation engine will prioritize these topics for personalization. Which means the site’s algorithm will start to funnel the user similar stories and pages—all to increase engagement.
engagement is the only metric that matters, as more engagement means more screen time to be exposed to advertisements. This is the darker side of Silicon Valley’s much celebrated metric of “user engagement.” By focusing so heavily on greater engagement, social media tends to parasitize our brain’s adaptive mechanisms. As it happens, the most engaging content on social media is often horrible or enraging. According to evolutionary psychologists, in order to survive in premodern times, humans developed a disproportionate attentiveness toward potential threats. The reason we instinctually pay
...more
The randomness of a slot machine prevents the player from being able to strategize or plan, so the only way to get a reward is to keep playing. The rewards are designed to be just frequent enough to reengage you after a losing streak and keep you going. In gambling, a casino makes money from the number of turns a player takes. On social media, a platform makes money from the number of clicks a user performs. This is why there are infinite scrolls on newsfeeds—there is very little difference between a user endlessly swiping for more content and a gambler pulling the slot machine lever over and
...more
Once a group reached a certain number of members, CA would set up a physical event. CA teams would choose small venues—a coffee shop or bar—to make the crowd feel larger.
In many ways, Gamergate created a conceptual framework for Bannon’s alt-right movement, as he knew there was an undercurrent populated by millions of intense and angry young men. Trolling and cyberbullying became key tools of the alt-right. But Bannon went deeper and had Cambridge Analytica scale and deploy many of the same tactics that domestic abusers and bullies use to erode stress resilience in their victims.
This framing of political correctness as an identity threat catalyzed a “boomerang” effect in people where counternarratives would actually strengthen, not weaken, the prior bias or belief. This means that when targets would see clips containing criticism of racist statements by candidates or celebrities, this exposure would have the effect of further entrenching the target’s racialized views, rather than causing them to question those beliefs.
anger interferes with information seeking.
people who displayed the JWH bias were, for example, more prone to victim-blaming in hypothetical scenarios of sexual assault. If the world is fair, then random bad things should not happen to innocent people, and therefore there must have been a fault in the victim’s behavior. Finding ways to blame victims is psychologically prophylactic for some people because it helps them cope with anxiety induced by uncontrollable environmental threats while maintaining a comforting view that the world will still be fair to them.
Steve Bannon aimed to affirm the ugliest biases in the American psyche and convince those who possessed them that they were the victims,
lacking the kind of traditional ethics review that is a prerequisite of academic research, there was never consideration of how this research could be misused—no one thought about how this could go wrong.
the Russian state sought to exploit this vulnerability—to hack American democracy. It would work, they decided, because American democracy is an inherently flawed system. The Russians created their self-fulfilling prophecy of social chaos by targeting and domesticating their propaganda to American citizens of similar worldviews, who would then click, like, and share. These narratives spread through a system of constitutionally protected free speech, and the U.S. government did nothing to stop them. Neither did Facebook.
As Cambridge Analytica identified, provoking anger and indignation reduced the need for full rational explanations and would put voters into a more indiscriminately punitive mindset.
some people would support the economy suffering if it meant that out-groups like metropolitan liberals or immigrants would suffer in the process—that,
And with social media companies not performing any checks on the advertising campaigns spreading throughout their platforms, there was no one standing guard to stop hostile entities seeking to sow chaos and disrupt our democracies.
Trump became the GOP nominee at the Republican National Convention on July 19. If my hunch was correct, Cambridge Analytica was not only using the data tool I had worked on to manipulate American voters into supporting him, it may have been knowingly or unknowingly working with Russians to sway the election.
Bannon was in a position to help arrange for Cambridge Analytica to get contracts with the U.S. government. Cambridge Analytica’s parent company, SCL Group, was already working on projects for the U.S. State Department. This meant that CA could access the U.S. government’s data, and vice versa. To my horror, I realized that Bannon could be creating his own private intelligence apparatus. And he was doing it for an administration that didn’t trust the CIA, the FBI, or the NSA. It felt like I was living in a nightmare.
Turnbull went further, revealing how Cambridge Analytica had set up the “crooked Hillary” narrative.
When Facebook found out that the story was about to break, it contacted Cambridge Analytica, which agreed to provide Facebook access to its servers and computers while the ICO was still in the process of requesting a warrant.
But what many Americans don’t understand is how connected the alt-right is. It is a coordinated global movement. And it became a massive security risk in 2016.
Why should Americans care about what Russia was doing in Britain? Because these Brexiteers shared the same data firm, in Cambridge Analytica, and the same adviser, in Steve Bannon, and they were clearly keeping the Russians informed at each step of the way. And these same Brexiteers were some of the very first people invited to Trump Tower after his surprise victory. The
ON MARCH 16, 2018, a day before The Guardian and The New York Times published my story, Facebook announced that it was banning me from not only Facebook but also Instagram. Facebook had refused to ban white supremacists, neo-Nazis, and other armies of hate, but it chose to ban me.
Facebook behaved as if it were a nation-state, rather than a company.
Users were told by Facebook that the enterprise was about bringing people together. But Facebook’s “community” was building separate neighborhoods just for people who look like them. As the platform watched them, read their posts, and studied how they interacted with their friends, its algorithms would then make decisions about how to classify users into digital neighborhoods of their kind—what Facebook called their “Lookalikes.” The reason for this, of course, was to allow advertisers to target these homogeneous Lookalikes with separate narratives just for people of their kind.
What was supposed to be so brilliant about the Internet was that people would suddenly be able to erode all those barriers and talk to anyone, anywhere. But what actually happened was an amplification of the same trends that took hold of a country’s physical spaces. People spend hours on social media, following people like them, reading news articles “curated” for them by algorithms whose only morality is click-through rates—articles that do nothing but reinforce a unidimensional point of view and take users to extremes to keep them clicking. What we’re seeing is a cognitive segregation, where
...more
We are all vulnerable to manipulation.
Over time, our biases can become amplified without our even realizing it. Many of us forget that what we see in our newsfeeds and our search engines is already moderated by algorithms whose sole motivation is to select what will engage us, not inform us. With most reputable news sources now behind paywalls, we are already seeing information inch toward becoming a luxury product in a marketplace where fake news is always free.
We risk creating a society obsessive about remembering, and we may have overlooked the value of forgetting, moving on, or being unknown.
What Cambridge Analytica did was use complex corporate setups across jurisdictions not only to launder money but to launder something that was becoming just as valuable: your data.

