World Without Mind: The Existential Threat of Big Tech

by Franklin Foer

The great sociologist Thorstein Veblen was obsessed with installing engineers in power and, in 1921, wrote a book making his case. His vision briefly became a reality. In the aftermath of World War I, American elites were aghast at all the irrational impulses unleashed by that conflict—the xenophobia, the racism, the urge to lynch and riot. What’s more, the realities of economic life had grown so complicated, how could politicians possibly manage them? Americans of all persuasions began yearning for the salvific ascendance of the most famous engineer of his time: Herbert Hoover. During the war, Hoover had organized a system that managed to feed starving Europe, despite the seeming impossibility of that assignment.

The Hoover experiment, in the end, hardly realized the happy fantasies about the Engineer King. A very different version of this dream, however, has come to fruition, in the form of the CEOs of the big tech companies.

we have begun to outsource our intellectual work to companies that suggest what we should learn, the topics we should consider, and the items we ought to buy.

THE PRECISE SOURCE OF FACEBOOK’S power is algorithms. That’s a concept repeated dutifully in nearly every story about the tech giants, yet it remains fuzzy at best to users of those sites. From the moment of the algorithm’s invention, it was possible to see its power, its revolutionary potential. The algorithm was developed in order to automate thinking, to remove difficult decisions from the hands of humans, to settle contentious debates. To understand the essence of the algorithm—and its utopian pretension—it’s necessary to travel back to its birthplace, the brain of one of history’s unimpeachable geniuses, Gottfried Leibniz.

Of all Leibniz’s schemes, the dearest was a new lexicon he called the universal characteristic—and it, too, sprang from his desire for peace.

What he imagined was an alphabet of human thought. It was an idea that he first pondered as a young student, the basis for his doctoral dissertation at Altdorf. Over the years, he fleshed out a detailed plan for realizing his fantasy. A group of scholars would create an encyclopedia containing the fundamental, incontestably true concepts of the world, of physics, philosophy, geometry, everything really. He called these core concepts “primitives,” and they would include things like the earth, the color red, and God. Each of the primitives would be assigned a numerical value, which allowed them to be combined to create new concepts or to express complex extant ones. And those numerical values would form the basis for a new calculus of thought, what he called the calculus ratiocinator. Leibniz illustrated his scheme with an example. What is a human? A rational animal, of course. That’s an insight that we can write like this: rational x animal = man But Leibniz translated this expression into an even more mathematical sentence. “Animal,” he suggested, might be represented with the number two; “rational” with the number three. Therefore: 2 x 3 = 6

“Once this has been done, if ever further controversies should arise, there should be no more reason for disputes between two philosophers than between two calculators.” Intellectual and moral argument could be settled with the disagreeing parties declaring, “Let’s calculate!” There would be no need for wars, let alone theological controversy, because truth would be placed on the terra firma of math.

He proposed a numeric system that used only zeros and ones, the very system of binary on which computing rests. He explained how automation of white-collar jobs would enhance productivity. But his critical insight was mechanical thinking, the automation of reason, the very thing that makes the Internet so miraculous, and the power of the tech companies so potentially menacing.

THOSE PROCEDURES THAT enable mechanical thinking came to have a name. They were dubbed algorithms.

This is different from equations, which have one correct result. Algorithms merely capture the process for solving a problem and say nothing about where those steps ultimately lead.

The process of translation, from concept to procedure to code, is inherently reductive. Complex processes must be subdivided into a series of binary choices. There’s no equation to suggest a dress to wear, but an algorithm could easily be written for that—it will work its way through a series of either/or questions (morning or night, winter or summer, sun or rain), with each choice pushing to the next.

An algorithm is a system, like plumbing or a military chain of command. It takes know-how, calculation, and creativity to make a system work properly. But some systems, like some armies, are much more reliable than others. A system is a human artifact, not a mathematical truism.

Algorithms upend the scientific method—the patterns emerge from the data, from correlations, unguided by hypotheses. They remove humans from the whole process of inquiry. Writing in Wired, Chris Anderson argued: “We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.”

Walmart’s algorithms found that people desperately buy strawberry Pop-Tarts as they prepare for massive storms. Still, even as an algorithm mindlessly implements its procedures—and even as it learns to see new patterns in the data—it reflects the minds of its creators, the motives of its trainers. Both Amazon and Netflix use algorithms to make recommendations about books and films.

Yet the algorithms make fundamentally different recommendations. Amazon steers you to the sorts of books that you’ve seen before. Netflix directs users to the unfamiliar.

The Harvard professor Latanya Sweeney conducted a study that found that users with African American names were frequently targeted with Google ads that bluntly suggested that they had arrest records in need of expunging. (“Latisha Smith, Arrested?”) Google is not particularly forthright about why such results appear. Their algorithm is a ferociously guarded secret. Yet, we know that Google has explicitly built its search engine to reflect values that it holds dear. It believes that the popularity of a Web site gives a good sense of its utility; it chooses to suppress pornography in its search results and not, say, anti-Semitic conspiracists; it believes that users will benefit from finding recent articles more than golden oldies. These are legitimate choices—and perhaps wise business decisions—but they are choices, not science.

But to do all these things, algorithms are constantly taking our measure. They make decisions about us and on our behalf. The problem is that when we outsource thinking to machines, we are really outsourcing thinking to the organizations that run the machines.

The company’s algorithms sort the thousands of things a Facebook user could possibly see down to a smaller batch of choice items. And then within those few dozen items, it decides what we might like to read first.

There’s a whole discipline, data science, to guide the writing and revision of algorithms. Facebook has a team, poached from academia, to conduct experiments on users. It’s a statistician’s sexiest dream—some of the largest data sets in human history, the ability to run trials on mathematically meaningful cohorts.

We know, for example, that Facebook sought to discover whether emotions are contagious. To conduct this trial, Facebook attempted to manipulate the mental state of its users. For one group, Facebook excised the positive words from the posts in the News Feed; for another group, it removed

the negative words. Each group, it concluded, wrote posts that echoed the mood of the posts it had reworded.

There’s no doubting the emotional and psychological power possessed by Facebook—at least Facebook doesn’t doubt it. It has bragged about how it increased voter turnout (and organ donation) by subtly amping up the social pressures that compel virtuous behavior. Facebook has even touted the results from these experiments in peer-reviewed journals: “It is possible that more of the .60% growth in turnout between 2006 and 2010 might have been caused by a single message on Facebook.” No other company has so precisely boasted about its ability to shape democracy like this—and for good reason. It’s too much power to entrust to a corporation.

Whether the information is true or concocted, authoritative reporting or conspiratorial opinion, doesn’t really seem to matter much to Facebook. The crowd gets what it wants and deserves.

Some news wires use algorithms to write stories about crime, baseball games, and earthquakes, the most rote journalistic tasks. Algorithms have produced fine art and composed symphonic music, or at least approximations of them. It’s a terrifying trajectory, especially for those of us in these lines of work. If algorithms can replicate the process of creativity, then there’s little reason to nurture human creativity.

Facebook would never put it this way, but algorithms are meant to erode free will, to relieve humans of the burden of choosing, to nudge them in the right direction. Algorithms fuel a sense of omnipotence, the condescending belief that our behavior can be altered, without our even being aware of the hand guiding us, in a superior direction. That’s always been a danger of the engineering mind-set, as it moves beyond its roots in building inanimate stuff and begins to design a more perfect social world. We are the screws and rivets in the grand design.

Organizing knowledge is an ancient pursuit. Those who toiled in this field over the centuries—librarians and bookstore owners, scholars and archivists—were trained to go about their work lovingly, almost worshipfully. A professional code implored them to treat their cargo as if the world depended on its safe transit through the generations. The tech companies share none of that concern. They have presided over the collapse of the economic value of knowledge, which has severely weakened newspapers, magazines, and book publishers.

It’s commonly argued that they don’t really deserve blame for this demise. According to this strain of conventional wisdom, it was inevitable that the price of knowledge would evaporate in the presence of the Internet. That narrative casts these companies as innocent bystanders, when, in fact, they were active, brutal accomplices. To build their empires, they targeted the weak economic underpinnings of knowledge and they knocked them right out. It was Jeff Bezos who pioneered this approach, even before the Internet had begun to truly take form—and he chose the most unlikely starting place.

Bezos and his boss, an eccentric computer scientist called David Shaw, even kicked around the idea of creating an “everything store”—a site that would serve as the mother of all intermediaries between the world’s manufacturers and its customers. But Bezos methodically studied the possibilities for commerce on the burgeoning medium, and considered that big idea a few steps early. Before consumers would shop at an everything store, they needed to acclimate to online shopping. He went searching for the ideal gateway product. The key would be to find a business that could easily be mastered by a small, underfunded operation—that would quickly build the trust of consumers, that didn’t require traveling the world to source inventory, that would allow for low-cost experimentation. After meticulous analysis, he decided that books—not office supplies, not music, not socks—were his best play.

The world’s digital trove of knowledge isn’t terribly useful without mechanisms for searching and sorting the ethereal holdings. That’s the trick Amazon—and the other knowledge monopolists—have managed. Amazon didn’t just create the world’s biggest bookstore; it made its store far more usable, far more efficient, than browsing the aisles of a Barnes and Noble or cruising a library’s card catalog. And beyond that, Amazon anticipated your desires, using its storehouse of data to recommend your next purchase, to strongly suggest a course for navigating knowledge.

Put differently, wealth was more likely to be hatched from computer code, a television series, a patent, a financial instrument. King Knowledge even determined the fruit of the soil. Take Monsanto, which produces the seeds that account for 80 percent of all corn and 90 percent of all soybeans grown in the United States. What Monsanto possesses, what it ferociously hoards, is the genetic traits of these seeds. Its comparative advantage isn’t factories, but laboratories.

If left to the devices of the market, the price of knowledge would quickly collapse, destroyed by the ease with which it could be freely copied. But the government doesn’t permit this collapse. One of its primary economic responsibilities is preserving the value of knowledge. It shelters the creators of knowledge from the rigors of the competitive marketplace, granting them a temporary state-sponsored monopoly in the form of patents and copyrights.

The culture had changed. Once an underground, amateur pastime, the bootlegging of intellectual property became an accepted business practice. Sites like the Huffington Post liberally plucked the best paragraphs of news stories, with a grudging link back to the original item. Google scanned every book it could find. Apple’s advertisements preached, “Rip, Mix, Burn—After all, it’s your music.” Larry Lessig, a law professor who served as a chief champion of this new era, declared: “The defining feature of the Internet is that it leaves resources free.” We could portray the changes as a matter of piracy—and there was surely a lot of that. But that wasn’t the significant development. Media accepted the economic collapse of knowledge, as if it were as irresistible as the weather.

This abundance of free material, however, created a new form of scarcity—with so much to read, see, and hear, with the unending web of links, it became almost impossible to grab an audience’s attention. David Foster Wallace called the condition Total Noise. With it, our reading became peripatetic, less focused.

Amazon organizes retail into a coherent, usable marketplace, to make no mention of it being the largest, most trafficked bookstore in human history; Google culls the entirety of the Web so that we have some sensible progression for considering its offerings; Facebook provides a directory of people, as well as a method for managing social lives. Without these tools, the Internet becomes unusable. “Searching and filtering are all that stand between this world and the Library of Babel,” the science writer James Gleick has argued.

THE BIG TECH COMPANIES didn’t just benefit from the economic collapse of knowledge. They maneuvered to shred the value of knowledge, so that old media would come to helplessly depend on their platforms.

Steve Jobs could have easily designed the iPod to make it inhospitable to stolen music. But he initially refused to build the iPod so that it would block unlicensed content. At the same time Jobs’s device enabled piracy, Jobs himself decried digital thievery. He was playing a cunning game: After helping push the music business to the brink, he would save it and come to dominate it. Eighteen months after creating the iPod, he debuted an online store, iTunes, that became the place where a vast percentage of all music was purchased. In the face of piracy, enfeebled producers lay prostrate in front of their new savior, even if Apple dismembered the once profitable album by selling individual songs for ninety-nine cents. From the wreckage Apple helped create, it built a new monopoly—60 percent of digital downloaded music is sold through iTunes—although streaming services have begun to weaken its decade-long grip on the business.

When newspapers and magazines require subscriptions to access their pieces, Google and Facebook tend to bury them;

Amazon doesn’t quite preach the same gospel, but it shares the same basic approach. It deflated the price of the books that it sells and made implicit arguments about their value.

When Facebook was assailed for abetting the onslaught of false news stories during the 2016 presidential campaign—a steady stream of fabricated right-wing conspiracies that boosted Donald Trump’s candidacy—Mark Zuckerberg initially disclaimed any culpability.

Though Zuckerberg denies it, the process of guiding the public to information is a source of tremendous cultural and political power. In the olden days, we described that power as gatekeeping—and it was a sacred obligation.

Jeff Bezos is the most populist of the tech CEOs, and the most strident critic of gatekeepers among them.

IN THE SUMMER OF 2013, Bezos bought the Washington Post. The transaction came as a shock to the elite system. For eight decades, the newspaper had been run by the Graham family—a clan that came to represent the noblest, most public-minded strain of American aristocracy, or at least that was their reputation in their own crust of society.

“GATEKEEPING,” AS A TERM APPLIED to media, entered the vernacular in the aftermath of World War II. After witnessing great cultures enthusiastically submit to fascism, American social scientists began to scour their own society for weaknesses. How did public opinion work in this country? What fascist tendencies were lurking on the crabgrass frontier? A rash of academic studies attempted to discern how information traveled to the common man, probing for points that demagogues might exploit. In the era—just before Edward R. Murrow and Walter Cronkite came to represent journalistic authority—newspapers were the nation’s information byway. They, therefore, were essential subjects for vigorous study. In a medieval village, the gatekeeper had the power to admit (or turn away) entrants into the communal sanctum. At a newspaper, it was the editor who played this part.

That’s a quaint thesis, the idea that certain well-placed individuals, full of conscious and submerged biases, exert control over the flow of information. But it’s also the truth.

But lofty ideals were transmitted from one generation of Grahams to the next. Before Don Graham could assume his birthright, he needed to intimately learn his city and its paper. He worked as a cop in the Ninth Precinct and as a sports editor, an apprenticeship in humility. The Grahams, to their credit, eventually acknowledged that their power required restraints beyond their own best intentions. A code of behavior governed the Post, as it did most major metropolitan dailies.

“Even well-meaning gatekeepers slow innovation.” This isn’t simply a slogan, it is a highly developed theory of history. The narrative goes like this: Once upon a time, the world needed gatekeepers. Resources were limited, so they had to be prudently rationed by enlightened elites. Scarcity, however, has now faded into the past thanks to the collapsing price of computing.

An obvious falsehood resides at the heart of Bezos’s account. He may have no wish to play the role of gatekeeper, but that’s exactly what he is. Yes, the old mode of gatekeeping excluded books from shelves and articles from magazines. Amazon, by contrast, sells nearly every cultural artifact produced by Western civilization. But let’s not confuse Amazon with a utopian experiment in participatory democracy. Amazon always gives better treatment to some artifacts than others—promoting them in email, on its home page, and through its recommendation algorithms. This is tremendous cultural power, especially given how so many of Amazon’s competitors have melted in the face of its size and prowess.

When Bezos debuted the Kindle, he surprised publishers by announcing that Amazon would sell e-books for $9.99—a sum that Bezos arbitrarily plucked and then blared to the public without giving the publishers any warning. This was brilliant gamesmanship. Bezos cemented a public impression about the value of e-books.

Amazon’s negotiating tactics with publishers are almost sadistic.

When sparring over terms with the publishing conglomerate Macmillan, it stripped the company’s books of the buttons that allow consumers to purchase them. In its dealings with Hachette, it delayed shipment of books. When cutting a deal with publishers, it doesn’t bother with innuendo. According to some who have sat across the table from Amazon, the company leaves no doubt that it will suppress a publisher’s performance in its algorithms and eliminate its books from its emails if the company rejects its terms.