Zucked: Waking Up to the Facebook Catastrophe

by Roger McNamee

And she concluded by apologizing only for Facebook’s poor communication. If Sheryl’s comments are any indication, running experiments on users without prior consent is a standard practice at Facebook.

Research by Cass Sunstein, who was the administrator of the White House Office of Information and Regulatory Affairs for the first Obama administration, indicates that when like-minded people discuss issues, their views

tend to get more extreme over time.

Facebook wants us to believe that it is merely a platform on which others act and that it is not responsible for what those third parties do. Both assertions warrant debate. In reality, Facebook created and operates a complex system built around a value system that increasingly conflicts with the values of the users it is supposed to serve. Where Facebook asserts that users control

their experience by picking the friends and sources that populate their News Feed, in reality an artificial intelligence, algorithms, and menus created by Facebook engineers control every aspect of that experience.

Facebook’s notion that a platform with more than two billion users can and should police itself also seems both naïve and self-serving, especially given

the now plentiful evidence to the contrary. Even if it were “just a platform,” Facebook has a responsibility for protecting users from harm. Deflection of responsibility has serious consequences.

By enabling anonymity and/or private Groups, the platforms removed the stigma, enabling like-minded people, including extremists, to find one another, communicate, and, eventually, to lose the fear of social stigma.

On the internet, even the most socially unacceptable ideas can find an outlet. As a proponent of free speech, I believe every person is entitled to speak his or her mind. Unfortunately, anonymity, the ability to form Groups in private, and the hands-off attitude of platforms have altered the normal

balance of free speech, often giving an advantage to extreme voices over reasonable ones. In the absence of limits imposed by the platform, hate sp...

They disregard expertise in favor of voices from their tribe.

They refuse to accept uncomfortable facts, even ones that are incontrovertible. This is how a large minority of Americans abandoned newspapers in favor of talk radio and websites that peddle conspiracy theories. Filter bubbles and preference bubbles undermine democracy by eliminating the

last vestiges of common ground among a huge percentage of Americans. The tribe is all that matters, and anything that advances the tribe is legitimate. You see this effect today among people whose embrace of Donald Trump ...

few years e...

The shared values that form the foundation of our democracy proved to be powerless against the preference bubbles that have evolved over the past decade. Facebook does not create preference bubbles, but it is the ideal incubator for them. The algorithms ensure that users who like one piece of disinformation will be fed more disinformation. Fed enough

disinformation, users will

eventually wind up first in a filter bubble and then in a preference bubble. If you are a bad actor and you want to manipulate people in a preference bubble, all you have to do is infiltrate the tribe, deploy the appropriate dog whistles, and you are good to g...

THE SAD TRUTH IS that Facebook and the other platforms are real-time systems with powerful tools optimized for behavior modification.

If you join the Group, the choice appears to be yours, but the reality is that Facebook planted

the seed. It does so not because conspiracy theories are good for you but because conspiracy theories are good for them.

Research suggests that people who accept one conspiracy theory have a high likelihood ...

Tristan’s insights jolted me, forcing me to accept the fact that Facebook, YouTube, and Twitter had created systems that modify user behavior. They should have realized that global scale

would have an impact on

the way people used their products and would raise the stakes for society. They should have anticipated violations of their terms of service and taken steps to prevent them. Once made aware of the interference, they should have cooperated with investigators. I could no longer pretend that Facebook was a victim. ...

The people at Facebook live in their own preference bubble. Convinced of the nobility of their mission, Zuck and his employees reject cr...

with the same approach that created the problem in the first place: more AI, more code, more short-term fixes. They do not do this because they are bad people. They do this because success has warped their perception of reality. To them, connecting 2.2 billion people is so obviously a good thing, and continued growth so important, that they cannot imagine that the problems that have resulted could be in any way linked to their designs or business decisions. It would never oc...

business. As a result, when confronted with evidence that disinformation and fake news spread over Facebook influenced the Brexit referendum in the United Kingdom and a presidential election in the United States, Facebook took steps that spoke volumes about the company’s world view. They demoted publishers in favor of family, friends, and Groups on the theory that information from those sources would be more trustworthy. The problem is that family, friends, and...

very disinformation and fake news that Facebook would...

Facebook would have you believe that the user is always in control, but as I have said before, user control is an illusion. Maintaining that illusion is central to every platform’s success, but with

Facebook, it is especially disingenuous. Menu choices limit user actions to things that serve Facebook’s interests.

Facebook’s terms of service have one goal and one goal only: to protect the company from legal liability.

By using the platform, we give Facebook permission to do just about anything it wants.

Worse yet, we behave as though notifications are personal to us, completely missing that they are automatically generated, often by an algorithm tied to an artificial intelligence that has concluded that the notification is just the thing to provoke an action that will serve the platform’s economic interests.

Everyone wants to feel approved of by others. We want our posts to be liked. We want people to respond to our texts, emails, tags, and shares. The need for social approval is what made Facebook’s Like button so powerful. By controlling how often a

user experiences social approval, as evaluated by others, Facebook can get that user to do things that generate billions of dollars in economic value. This makes sense because the currency of Facebook is attention. Users manicure their image in the hope of impressing others, but they soon discover that the best way to get attention is through emotion and conflict. Want attention online? Say something outrageous. This phenomenon first emerged decades ago in online forums such as The WELL, which often devolved into mean-spirited confrontation and has reappeared in every generation

of tech platform si...

when a person does something for us, we feel obligated to reciprocate. When someone “follows” us on

Instagram, we feel obligated to “follow” them in return. When we see an “Invitation to Connect” on LinkedIn from a friend, we may feel guilty if we do not reciprocate the gesture and accept it. It feels organic, but it is not.

Tagging was a game changer for Facebook because photos are one of the main reasons users

visit Facebook daily in the first place. Each tagged photo brings with it a huge trove of data and metadata about location, activity, and friends, all of which can be used to target ads more effectively. Thanks to photo tagging, users have built a giant database of photos for Facebook, complete with all the information

it effectively. Other platforms play this game, too, but not ...

FOMO and the need for social approval combine to magnify the already stressful social lives of teenagers. Teenagers are particularly vulnerable to social pressure, and internet platforms add complexity to that equation that we are only beginning to understand.

NONE OF US WANTS to admit to any addiction. We like to think of ourselves as being in control.

I did not understand my behavioral addiction until I joined forces with

Tristan in April 2017. Until then, I thought I bore sole responsibility for the problem. I assumed that only people with my heat-seeking love for technology could fall victim to unhelpful tech behaviors. Tristan opened my eyes to the reality that technology companies had devoted some of their

best minds to exploiting the weaknesses in human psychology. They did so on purpose. To make money. And when they had made themselves ridiculously wealthy, they kept doing it because it never occurred to them to do anything else. ...

from shareholders. Given that the founders of both Facebook and Google have total control of their companies,...

us can resist the lure of persuasive technology. All we can do is minimize the stimulus, or avoid it altoge...

element of persuasive technology is a way to...

From its earliest days, the internet’s culture advocated free speech and anonymity without constraint. At small scale, such freedom of speech was liberating, but the architects of the World Wide Web failed