More on this book
Community
Kindle Notes & Highlights
by
Jaron Lanier
Read between
October 5 - October 16, 2020
Now the real news is called “fake news,” because by the standards of BUMMER, what is real is fake; in BUMMER, reality has been replaced by stupid numbers.
Podcasts still rely on stores and subscriptions, so they maintain a person-to-person structure instead of a person-to-crowd/algorithm/hidden-manipulator structure. Enjoy podcasts while you can. Please stay alert, and if podcasts are ruined, stop making them and stop listening. For now, remember that you have only the most tenuous connection to the meaning of the stuff you add to the BUMMER monster.
Recall that Component C of BUMMER—Cramming experiences into your life—means that algorithms determine what you see. That means you don’t know what other people are seeing, because Component C is calculating different results for them. You can’t know how much the worldviews of other people are being biased and shaped by BUMMER. Personalized search, feeds, streams, and so on are at the root of this problem.
A wonderful way to notice social perception is to travel to a country where you don’t speak the language. You’ll find that you are suddenly very attuned to what other people are doing and what they are paying attention to, because that’s the only way to know what’s going on. One time I noticed people in a jungle in Thailand paying attention to a certain direction, so I did, too, just in time to get out of the way of speeding army jeeps that came out of nowhere. Social perception saved my life. It has always been part of how humanity has survived. But when we’re all seeing different, private
...more
The speed, idiocy, and scale of false social perceptions have been amplified to the point that people often don’t seem to be living in the same world, the real world, anymore.
Can you imagine if Wikipedia showed different versions of entries to each person on the basis of a secret data profile of that person? Pro-Trump visitors would see an article completely different from the one shown to anti-Trump people, but there would be no accounting of all that was different or why. This might sound dystopian or bizarre, but it’s similar to what you see in your BUMMER feed. Content is chosen and ads are customized to you, and you don’t know how much has been changed for you, or why.
Another way to see the problem is to think about public spaces. If you share a space with people who aren’t looking at their smartphones, you are all in that space together. You have a common base of experience. It can be an amazing feeling, and it’s a big reason why people go to clubs, sports events, and houses of worship.
Empathy2 is the fuel that runs a decent society. Without it, only dry rules and competitions for power are left.
A common and correct criticism of BUMMER is that it creates “filter bubbles.”3 Your own views are soothingly reinforced, except when you are presented with the most irritating versions of opposing views, as calculated by algorithms. Soothe or savage: whatever best keeps your attention.
to review, the term should be “manipulate,” not “engage,” since it’s done in the service of unknown third parties who pay BUMMER companies to change your behavior. Otherwise, what are they paying for? What else could Facebook say it’s being paid tens of billions of dollars to do?)
While no one outside Facebook—or maybe even inside Facebook—knows how common or effective dark ads and similar messages have been,6 the most common form of online myopia is that most people can only make time to see what’s placed in front of them by algorithmic feeds. I fear the subtle algorithmic tuning of feeds more than I fear blatant dark ads.
The results are tiny changes in the behavior of people over time. But small changes add up, like compound interest. This is one reason that BUMMER naturally promotes tribalism and is tearing society apart, even if the techies in a BUMMER company are well-meaning. In order for BUMMER code to self-optimize, it naturally and automatically seizes upon any latent tribalism and racism, for these are the neural hashtags waiting out there in everyone’s psyche, which can be accentuated for the purpose of attention monopoly.
The version of the world you are seeing is invisible to the people who misunderstand you, and vice versa.
The ability to theorize about what someone else experiences as part of understanding that person is called having a theory of mind. To have a theory of mind is to build a story in your head about what’s going on in someone else’s head. Theory of mind is at the core of any sense of respect or empathy, and it’s a prerequisite to any hope of intelligent cooperation, civility, or helpful politics. It’s why stories exist.
When you can only see how someone else behaves, but not the experiences that influenced their behavior, it becomes harder to have a theory of mind about that person. If you see someone hit someone else, for instance, but you did not see that they did it in defense of a child, you might misinterpret what you see. In the same way, if you don’t see the dark ads, the ambient whispers, the cold-hearted memes, and the ridicule-filled customized feed that someone else sees, that person will just seem crazy to you. And that is our new BUMMER world. We seem crazy to each other, because BUMMER is
...more
Trump supporters seem nuts to me, and they say liberals seem nuts to them. But it’s wrong to say we’ve grown apart and can’t understand each other. What’s really going on is that we see less than ever before of what others are seeing, so we have less opportunity to understand each other.
The opacity of our times is even worse than it might be because the degree of opacity is itself opaque. I remember when the internet was supposed to bring about a transparent society. The reverse has happened.
Research4 shows a world that is not more connected,5 but instead suffers from a heightened sense of isolation.
Facebook researchers have practically bragged9 that they could make people unhappy without the people realizing why.10 Why promote something like that as a great research result? Wouldn’t it be damaging to Facebook’s brand image? The reason might have been that it was great publicity for reaching the true customers, those who pay to manipulate. The ones who are manipulated, meaning you, are the product, not the customer.
Yes, of course it’s great that people can be connected,12 but why must they accept manipulation by a third party as the price of that connection? What if the manipulation, not the connection, is the real problem?
Why the variety? Wouldn’t one way of bumming people out be enough? Since the core strategy of the BUMMER business model is to let the system adapt automatically to engage you as much as possible, and since negative emotions can be utilized more readily, of course such a system is going to tend to find a way to make you feel bad. It will dole out sparse charms14 in between the doldrums as well, since the autopilot that tugs at your emotions will discover that the contrast between treats and punishment is more effective than either treats or punishment alone. Addiction is associated with
...more
Based on the research, there are trends in the forms that unhappiness takes, so I could guess about what’s going on with you. You might have less sex than you seek in proportion to the amount of time you use apps to seek sex.16 You’re sitting there swiping at a screen. You might spend less time with your family in proportion to the cuteness of the presentation of your family life you put out there on social media.17 You might be at risk for self-harm in proportion to your social media use, especially if you’re a young woman.18 You might be making traumatic experiences worse by using social
...more
What bums me out is not some particular surface pattern—like seeing everyone else misrepresent their lives as being more wealthy, happy, and trouble-free than they are—but instead it’s the core BUMMER system. Being addicted and manipulated makes me feel bad, but there’s more to it than that. BUMMER makes me feel judged within an unfair and degrading competition, and to no higher purpose.
Even with ancient services like early Usenet, I found that there was a strange, unfamiliar hollow in me after a session. It was something I had not felt since I was a child. An insecurity, a feeling of not making the grade, a fear of rejection, out of nowhere.
Here is one thing I discovered about myself: I don’t mind being judged if the judges put in real effort, and a higher purpose is being honestly served, but I really don’t like it when a crowd judges me casually, or when a stupid algorithm has power over me.
BUMMER algorithms must put you into categories and rank you in order to do anything BUMMER at all. The whole purpose of BUMMER is turning you and changes to your behavior into a product. The algorithms fundamentally work to favor platform owners and advertisers, and those parties need abstractions of you in order to manipulate you.
The BUMMER algorithms behind companies like Facebook and Google are stored in some of the few files in the world that can’t be hacked; they’re kept that secret. The deepest secrets of the NSA21 and the CIA22 have leaked, repeatedly, but you can’t find a copy of Google’s search algorithm or Facebook’s feed algorithm on the dark web.