You Are Not A Gadget

by Jaron Lanier

there is a new kind of manifest destiny that provides us with a mission to accomplish. The meaning of life, in this view, is making the digital system we call reality function at ever-higher “levels of description.”

A web page is thought to represent a higher level of description than a single letter, while a brain is a higher level than a web page. An increasingly common extension of this notion is that the net as a whole is or soon will be a higher level than a brain.

The real function of the feature isn’t to make life easier for people. Instead, it promotes a new philosophy: that the computer is evolving into a life-form that can understand people better than people can understand themselves.

I say that information doesn’t deserve to be free.

if the bits can potentially mean something to someone, they can only do so if they are experienced. When that happens, a commonality of culture is enacted between the storer and the retriever of the bits. Experience is the only process that can de-alienate information.

Information of the kind that purportedly wants to be free is nothing but a shadow of our own minds, and wants nothing on its own. It will not suffer if it doesn’t get what it wants.

He applied one of the first computers to break a Nazi secret code, called Enigma, which Nazi mathematicians had believed was unbreakable.

you have to remember that before computers came along, the steam engine was a preferred metaphor for understanding human nature.

The common use of computers, as we understand them today, as sources for models and metaphors of ourselves is probably about as reliable as the use of the steam engine was back then.

Shortly before his death, he presented the world with a spiritual idea, which must be evaluated separately from his technical achievements. This is the famous Turing test.

Can the judge tell which is the man? If not, is the computer conscious? Intelligent? Does it deserve equal rights?

But the Turing test cuts both ways. You can’t tell if a machine has gotten smarter or if you’ve just lowered your own standards of intelligence to such a degree that the machine seems smart.

Wikipedia, for instance, works on what I call the Oracle illusion, in which knowledge of the human authorship of a text is suppressed in order to give the text superhuman validity. Traditional holy books work in precisely the same way and present many of the same problems.

Computers fascinate and frustrate us in a similar way. Children can learn to program them, yet it is extremely difficult for even the most accomplished professional to program them well.

Whenever a computer is imagined to be intelligent, what is really happening is that humans have abandoned aspects of the subject at hand in order to remove from consideration whatever the computer is blind to. This happened to chess itself in the case of the Deep Blue–Kasparov tournament.

The most important thing to ask about any technology is how it changes people. And in order to ask that question I’ve used a mental device called the “circle of empathy” for many years.

An imaginary circle of empathy is drawn by each person. It circumscribes the person at some distance, and corresponds to those things in the world that deserve empathy.

If someone falls within your circle of empathy, you wouldn’t want to see him or her killed. Something that is clearly outside the circle is fair game.

When you change the contents of your circle, you change your conception of yourself. The center of the circle shifts as its perimeter is changed. The liberal impulse is to expand the circle, while conservatives tend to want to restrain or even contract the circle.

In the case of slavery, it turned out that, given a chance, slaves could not just speak for themselves, they could speak intensely and well.

The new twist in Silicon Valley is that some people—very influential people—believe they are hearing algorithms and crowds and other internet-supported nonhuman entities speak for themselves. I don’t hear those voices, though—and I believe those who do are fooling themselves.

If you try to pretend to be certain that there’s no mystery in something like consciousness, the mystery that is there can pop out elsewhere in an inconvenient way and ruin your objectivity as a scientist.

Consciousness is situated in time, because you can’t experience a lack of time, and you can’t experience the future.

Zombies are familiar characters in philosophical thought experiments. They are like people in every way except that they have no internal experience. They are unconscious, but give no externally measurable evidence of that fact.

If the books in the cloud are accessed via user interfaces that encourage mashups of fragments that obscure the context and authorship of each fragment, there will be only one book.

Authorship—the very idea of the individual point of view—is not a priority of the new ideology.

individuals can author books or blogs or whatever, but people are encouraged by the economics of free content, crowd dynamics, and lord aggregators to serve up fragments instead of considered whole expressions or arguments. The efforts of authors are appreciated in a manner that erases the boundaries between them.

Any singular, exclusive book, even the collective one accumulating in the cloud, will become a cruel book if it is the only one available.

We know a little about what Aztec or Inca music sounded like, for instance, but the bits that were trimmed to make the music fit into the European idea of church song were the most precious bits.

Something like missionary reductionism has happened to the internet with the rise of web 2.0. The strangeness is being leached away by the mush-making process.

fashionable idea in technical circles is that quantity not only turns into quality at some extreme of scale, but also does so according to principles we already understand.

I disagree. A trope from the early days of computer science comes to mind: garbage in, garbage out.

there is no evidence that quantity becomes quality in matters of human expression or achievement. What matters instead, I believe, is a sense of focus, a mind in effective concentration, and an adventurous individual imagination that is distinct from the crowd.

All that the social networking services offer is a prod to use the web in a particular way, according to a particular philosophy.

Enlightened designers leave open the possibility of either metaphysical specialness in humans or in the potential for unforeseen creative processes that aren’t explained by ideas like evolution that we already believe we can capture in software systems. That kind of modesty is the signature quality of being human-centered.

the customers of social networks are not the members of those networks. The real customer is the advertiser of the future,

The hope of a thousand Silicon Valley start-ups is that firms like Facebook are capturing extremely valuable information called the “social graph.”

When Facebook has attempted to turn the social graph into a profit center in the past, it has created ethical disasters. A famous example was 2007’s Beacon.

When a Facebook user made a purchase anywhere on the internet, the event was broadcast to all the so-called friends in that person’s network.

The only hope for social networking sites from a business point of view is for a magic formula to appear in which some method of violating privacy and dignity becomes acceptable.

The term “wisdom of crowds” is the title of a book by James Surowiecki and is often introduced with the story of an ox in a marketplace. In the story, a bunch of people all guess the animal’s weight, and the average of the guesses turns out to be generally more reliable than any one person’s estimate.

The reason the collective can be valuable is precisely that its peaks of intelligence and stupidity are not the same as the ones usually displayed by individuals. What makes a market work, for instance, is the marriage of collective and individual intelligence. A marketplace can’t exist only on the basis of having prices determined by competition. It also needs entrepreneurs to come up with the products that are competing in the first place.

Collectives can be just as stupid as any individual—and, in important cases, stupider. The interesting question is whether it’s possible to map out where the one is smarter than the many.

Signal processing is a bag of tricks engineers use to tweak flows of information.

One service performed by representative democracy is low-pass filtering, which is like turning up the bass and turning down the treble. Imagine the jittery shifts that would take place if a wiki were put in charge of writing laws.

Such chaos can be avoided in the same way it already is, albeit imperfectly: by the slower processes of elections and court proceedings. These are like bass waves.

The “wisdom of crowds” effect should be thought of as a tool. The value of a tool is its usefulness in accomplishing a task. The point should never be the glorification of the tool.

Nassim Nicholas Taleb has argued that applications of statistics, such as crowd wisdom schemes, should be divided into four quadrants. He defines the dangerous “Fourth Quadrant” as comprising problems that have both complex outcomes and unknown distributions of outcomes. He suggests making that quadrant taboo for crowds.

the user interface designs that arise from the ideology of the computing cloud make people—all of us—less kind. Trolling is not a string of isolated incidents, but the status quo in the online world.

We evolved to be both loners and pack members. We are optimized not so much to be one or the other, but to be able to switch between them.