The Age of Surveillance Capitalism

by Shoshana Zuboff

The uncontract is a feature of the larger complex that is the means of behavioral modification, and it is therefore an essential modality of surveillance capitalism. It contributes to economies of action by leveraging proprietary behavioral surplus to preempt and foreclose action alternatives, thus replacing the indeterminacy of social processes with the determinism of programmed machine processes. This is not the automation of society, as some might think, but rather the replacement of society with machine action dictated by economic imperatives.

Imagine you have a hammer. That’s machine learning. It helped you climb a grueling mountain to reach the summit. That’s machine learning’s dominance of online data. On the mountaintop you find a vast pile of nails, cheaper than anything previously imaginable. That’s the new smart sensor tech. An unbroken vista of virgin board stretches before you as far as you can see. That’s the whole dumb world. Then you learn that any time you plant a nail in a board with your machine learning hammer, you can extract value from that formerly dumb plank. That’s data monetization. What do you do? You start hammering like crazy and you never stop, unless somebody makes you stop. But there is nobody up here to make us stop. This is why the “internet of everything” is inevitable.

Inevitabilism precludes choice and voluntary participation. It leaves no room for human will as the author of the future. This raises questions: At what point does inevitabilism’s claim to ubiquitous extraction and execution shade into abuse? Will inevitabilism’s utopian declarations summon new forms of coercion designed to quiet restless populations unable to quell their hankering for a future of their choice?72

In effect, what we’re doing is replicating the digital experience in physical space.… So ubiquitous connectivity; incredible computing power including artificial intelligence and machine learning; the ability to display data; sensing, including cameras and location data as well as other kinds of specialized sensors.… We fund it all… through a very novel advertising model.… We can actually then target ads to people in proximity, and then obviously over time track them through things like beacons and location services as well as their browsing activity.83

It is eloquent testimony to the health care system’s failure to serve the needs of second-modernity individuals that we now access health data and advice from our phones while these pocket computers aggressively access us. M-health has triggered an explosion of rendition and behavioral surplus capture as individuals turn in record numbers to their fitness bands and diet apps for support and guidance.47 By 2016, there were more than 100,000 mobile health apps available on the Google Android and Apple iOS platforms, double the number in 2014.48 These rich data can no longer be imagined as cloistered within the intimate closed loops between a patient and her doctor or between an application and its dieters or runners. That bucolic vision has its holdouts, to be sure, but for surveillance capitalists this vision is but a faded daguerreotype.

Currently, the Illinois Biometric Privacy Act offers the most comprehensive legal protections, requiring companies to obtain written consent before collecting biometric information from any individual and, among other stipulations, granting individuals the right to sue a company for unauthorized rendition.53

There was a time when you searched Google, but now Google searches you.

In this future, children learn the principles of the One Voice—a run time, a new interface. It is available everywhere to execute their commands, anticipate their desires, and shape their possibilities. The omnipresence of the One Voice, with its fractious, eager marketplace-of-you concealed under its skirts, changes many things. Intimacy as we have known it is compromised, if not eliminated. Solitude is deleted. The children will learn first that there are no boundaries between self and market. Later they will wonder how it could ever have been different.

Instead of analyzing the content of user lists, such as favorite TV shows, activities, and music, they learned that simple “meta-data”—such as the amount of information shared—“turned out to be much more useful and predictive than the original raw data.” The computations produced on the strength of these behavioral metrics, when combined with automated linguistic analysis and internal Facebook statistics, led the research team to conclude that “we can predict a user’s score for a personality trait to within just more than one-tenth of its actual value.”51

Their discussion specifically cited Facebook CEO Mark Zuckerberg’s unilateral upending of established privacy norms in 2010, when he famously announced that Facebook users no longer have an expectation of privacy. Zuckerberg had described the corporation’s decision to unilaterally release users’ personal information, declaring, “We decided that these would be the social norms now, and we just went for it.”55

Personality analysis for commercial advantage is built on behavioral surplus—the so-called meta-data or mid-level metrics—honed and tested by researchers and destined to foil anyone who thinks that she is in control of the “amount” of personal information that she reveals in social media. In the name of, for example, affordable car insurance, we must be coded as conscientious, agreeable, and open. This is not easily faked because the surplus retrieved for analysis is necessarily opaque to us. We are not scrutinized for substance but for form. The price you are offered does not derive from what you write about but how you write it. It is not what is in your sentences but in their length and complexity, not what you list but that you list, not the picture but the choice of filter and degree of saturation, not what you disclose but how you share or fail to, not where you make plans to see your friends but how you do so: a casual “later” or a precise time and place? Exclamation marks and adverb choices operate as revelatory and potentially damaging signals of your self.

The findings showed that personality information predicted the likelihood of responses. People whom the machines rated as moral, trusting, friendly, extroverted, and agreeable tended to respond, compared to low response rates from people rated as cautious and anxious. Many of the characteristics that we try to teach our children and model in our own behavior are simply repurposed as dispossession opportunities for hidden machine processes of rendition. In this new world, paranoia and anxiety function as sources of protection from machine invasion for profit. Must we teach our children to be anxious and suspicious?

Varian awards surveillance capitalists the privilege of the experimenter’s role, and this is presented as another casual fait accompli. In fact, it reflects the final critical step in surveillance capitalists’ radical self-dealing of new rights. In this phase of the prediction imperative, surveillance capitalists declare their right to modify others’ behavior for profit according to methods that bypass human awareness, individual decision rights, and the entire complex of self-regulatory processes that we summarize with terms such as autonomy and self-determination.

Facebook’s surplus is aimed at solving one problem: how and when to intervene in the state of play that is your daily life in order to modify your behavior and thus sharply increase the predictability of your actions now, soon, and later. The challenge for surveillance capitalists is to learn how to do this effectively, automatically, and, therefore, economically, as a former Facebook product manager writes: Experiments are run on every user at some point in their tenure on the site. Whether that is seeing different size ad copy, or different marketing messages, or different call-to-action buttons, or having their feeds generated by different ranking algorithms.… The fundamental purpose of most people at Facebook working on data is to influence and alter people’s moods and behavior. They are doing it all the time to make you like stories more, to click on more ads, to spend more time on the site. This is just how a website works, everyone does this and everyone knows that everyone does this.7

The team celebrated its work as “some of the first experimental evidence to support the controversial claims that emotions can spread throughout a network,” and they reflected on the fact that even their relatively minimal manipulation had a measurable effect, albeit a small one.10

The salience of self-awareness as a bulwark against self-regulatory failure is also underscored in the work of two Cambridge University researchers who developed a scale to measure a person’s “susceptibility to persuasion.” They found that the single most important determinant of one’s ability to resist persuasion is what they call “the ability to premeditate.”23 This means that people who harness self-awareness to think through the consequences of their actions are more disposed to chart their own course and are significantly less vulnerable to persuasion techniques. Self-awareness also figures in the second-highest-ranking factor on their scale: commitment. People who are consciously committed to a course of action or set of principles are less likely to be persuaded to do something that violates that commitment.

Most pointedly, Facebook’s declaration of experimental authority claims surveillance capitalists’ prerogatives over the future course of others’ behavior. In declaring the right to modify human action secretly and for profit, surveillance capitalism effectively exiles us from our own behavior, shifting the locus of control over the future tense from “I will” to “You will.” Each one of us may follow a distinct path, but economies of action ensure that the path is already shaped by surveillance capitalism’s economic imperatives. The struggle for power and control in society is no longer associated with the hidden facts of class and its relationship to production but rather by the hidden facts of automated engineered behavior modification.

The democratic impulse of US society, repelled by the excesses of its intelligence agencies, their support of criminal activities undertaken by the Nixon administration, and the migration of behavior modification as a means of disciplinary control in state institutions, led to the rejection of behavioral modification as an extension of governmental power. Unknown to the senators, scholars, rights activists, litigators, and many other citizens who stood against the antidemocratic incursions of the behavioral engineering vision, these methods had not died. The project would resurface in a wholly unexpected incarnation as a creature of the market, its unprecedented digital capabilities, scale, and scope now flourishing under the flag of surveillance capitalism. During the same years that US democratic forces combined to resist behavior modification as a form of state power, the energies of capitalist counterinsurgency were already at work within society. The corporation was to enjoy the rights of personhood but be free of democratic obligations, legal constraints, moral calculations, and social considerations. Certainly in the US case, a weakened state in which elected officials depend upon corporate wealth in every election cycle has shown little appetite to contest behavior modification as a market project, let alone to stand for the moral imperatives of the autonomous individual.

In its latest incarnation, behavioral modification comes to life as a global digital market architecture unfettered by geography, independent of constitutional constraints, and formally indifferent to the risks it poses to freedom, dignity, or the sustenance of the liberal order that Ervin’s subcommittee was determined to defend. This contrast is even more distressing in light of the fact that in the mid-twentieth century, the means of behavior modification were aimed at individuals and groups who were construed as “them”: military enemies, prisoners, and other captives of walled disciplinary regimes. Today’s means of behavioral modification are aimed unabashedly at “us.” Everyone is swept up in this new market dragnet, including the psychodramas of ordinary, unsuspecting fourteen-year-olds approaching the weekend with anxiety. Every avenue of connectivity serves to bolster private power’s need to seize behavior for profit. Where is the hammer of democracy now, when the threat comes from your phone, your digital assistant, your Facebook login? Who will stand for freedom now, when Facebook threatens to retreat into the shadows if we dare to be the friction that disrupts economies of action that have been carefully, elaborately, and expensively constructed to exploit our natural empathy, elude our awareness, and circumvent our prospects for self-determination? If we fail to take notice now, how long before we are numb to this incursion and to all the incursions? How long unt...

Now we know that surveillance capitalists’ ability to evade our awareness is an essential condition for knowledge production.

We choose the fallibility of shared promises and problem solving over the certain tyranny imposed by a dominant power or plan because this is the price we pay for the freedom to will, which founds our right to the future tense. In the absence of this freedom, the future collapses into an infinite present of mere behavior, in which there can be no subjects and no projects: only objects. In the future that the surveillance capitalism prepares for us, my will and yours threaten the flow of surveillance revenues. Its aim is not to destroy us but simply to author us and to profit from that authorship. Such means have been imagined in the past, but only now are they feasible. Such means have been rejected in the past, but only now have they been allowed to root. We are ensnared without awareness, shorn of meaningful alternatives for withdrawal, resistance, or protection.

“The commodity fiction disregarded the fact that leaving the fate of soil and people to the market would be tantamount to annihilating them.”22

If we are to rediscover our sense of astonishment, then let it be here: if industrial civilization flourished at the expense of nature and now threatens to cost us the Earth, an information civilization shaped by surveillance capitalism will thrive at the expense of human nature and threatens to cost us our humanity.

Industrial capitalism was marked by the specialized division of labor, with its historically specific characteristics: the conversion of craft work to mass production based on standardization, rationalization, and the interchangeability of parts; the moving assembly line; volume production; large populations of wage earners concentrated in factory settings; professionalized administrative hierarchies; managerial authority; functional specialization; and the distinction between white-collar work and blue-collar work. The list is illustrative, not exhaustive, but enough to remind us that industrial civilization was drawn from these expressions of the economic imperatives that ruled industrial expansion. The division of labor shaped culture, psychology, and social experience. The shift from craft to hourly wages created new populations of employees and consumers, men and women wholly dependent on the means of production owned and operated by private firms.

At a time when surveillance capitalism has emerged as the dominant form of information capitalism, we must ask the question: what kind of civilization does it foretell?

In reality, he writes, freedom and ignorance are synonyms. The acquisition of knowledge is heroic in that it rescues us from ignorance, but it is also tragic because it necessarily reveals the impossibility of freedom.

That entire work was trained on what Skinner continued to regard as the chief impediment to social progress: the conceptual confusion that cloaks our deepest ignorance in the sacred robes of freedom and dignity. Skinner argued that our allegiance to these lofty notions is simply the way we protect ourselves from the hard truths of “unsuspected controlling relations between behavior and environment.”56 They are a psychological “escape route” that slowly closes “as new evidences of the predictability of human behavior are discovered. Personal exemption from a complete determinism is revoked as a scientific analysis progresses… the achievements for which a person himself is to be given credit seem to approach zero… the behavior we admire is therefore the behavior we cannot yet explain.”57

any action regarded as an expression of free will is simply one for which “the vortex of stimuli” that produced it cannot yet be adequately specified. We merely lack the means of observation and calculation.

Big Brother’s vigilance is not restricted to the grand continents of armies and statecraft or the observable flows of bodies and crowds. Big Brother is a panvasive consciousness that infects and possesses each individual soul, displacing all attachments once formed in romantic love and good fellowship. The essence of its operation is not simply that it knows every thought and feeling but rather the ruthless tenacity with which it aims to annihilate and replace unacceptable inward experience. “We are not content with negative obedience, nor even with the most abject submission,” the cunning henchman O’Brien tells the rebellious Winston: We shall crush you down to the point from which there is no coming back.… When finally you surrender to us, it must be of your own free will. We do not destroy the heretic because he resists us: so long as he resists us we never destroy him. We convert him, we capture his inner mind, we reshape him.… We bring him over to our side, not in appearance, but genuinely, heart and soul. We make him one of ourselves before we kill him.71

Skinner understood his utopia as a methodological cure for the nightmare of crushed souls, a cure that, he insisted, was superior to any of the conventional political, economic, or spiritual remedies on offer. He scoffed at the notion that “democracy” held the solution because it is a political system that merely perpetrates the illusion of freedom while impeding the dominion of science. The promise of the “free market” as the curative for postwar society was an equally empty dream, he believed, because it rewards destructive competitiveness between people and classes. Skinner also rejected existentialism’s new humanism, seeing it as a breeding ground for passivity, and he regarded religion as the worst cure of all, enshrining ignorance and crippling the advance of science. Skinner’s cure was different and unique: a utopia of technique that promised a future of social equality and dispassionate harmony founded on the viewpoint of the Other-One, the “organism among organisms,” as the object of “behavioral engineering.” It is the future of Meyer’s dreams, in which Frazier, the founder-leader of the fictional Walden Two and Skinner’s outspoken avatar, describes that ideal community as a “superorganism” that can be shaped and controlled “as smoothly and efficiently as champion football teams.”73

Until the rise of surveillance capitalism, the prospect of instrumentarian power was relegated to a gauzy world of dream and delusion. This new species of power follows the logic of Planck, Meyer, and Skinner in the forfeit of freedom for knowledge, but those scientists each failed to anticipate the actual terms of this surrender. The knowledge that now displaces our freedom is proprietary. The knowledge is theirs, but the lost freedom belongs solely to us.

Forget the cliché that if it’s free, “You are the product.” You are not the product; you are the abandoned carcass. The “product” derives from the surplus that is ripped from your life.

Social trust is highly correlated with peaceful collective decision making and civic engagement. In its absence, the authority of shared values and mutual obligations slips away. The void that remains is a loud signal of societal vulnerability. Confusion, uncertainty, and distrust enable power to fill the social void. Indeed, they welcome it.

Totalitarianism seeks totality as a political condition and relies on violence to clear its path. Instrumentarianism seeks totality as a condition of market dominance, and it relies on its control over the division of learning in society, enabled and enforced by Big Other, to clear its path. The result is the application of instrumentarian power to societal optimization for the sake of market objectives: a utopia of certainty.

When driving, people mostly learn from their own mistakes, but they rarely learn from the mistakes of others. People collectively make the same mistakes over and over again. As a result, hundreds of thousands of people die worldwide every year in traffic collisions. AI evolves differently. When one of the self-driving cars makes an error, all of the self-driving cars learn from it. In fact, new self-driving cars are “born” with the complete skill set of their ancestors and peers. So collectively, these cars can learn faster than people. With this insight, in a short time self-driving cars safely blended onto our roads alongside human drivers, as they kept learning from each other’s mistakes.… Sophisticated AI-powered tools will empower us to better learn from the experiences of others.… The lesson with self-driving cars is that we can learn and do more collectively.33 This is a succinct but extraordinary statement of the machine template for the social relations of an instrumentarian society. The essence of these facts is that first, machines are not individuals, and second, we should be more like machines. The machines mimic each other, and so must we. The machines move in confluence, not many rivers but one, and so must we. The machines are each structured by the same reasoning and flowing toward the same objective, and so must we be structured.

In this way, the machine hive becomes the role model for a new human hive in which we march in peaceful unison toward the same direction based on the same “correct” understanding in order to construct a world free of mistakes, accidents, and random messes. In this world the “correct” outcomes are known in advance and guaranteed in action. The same ubiquitous instrumentation and transparency that define the machine system must also define the social system, which in the end is simply another way of describing the ground truth of instrumentarian society. In this human hive, individual freedom is forfeit to collective knowledge and action. Nonharmonious elements are preemptively targeted with high doses of tuning, herding, and conditioning, including the full seductive force of social persuasion and influence. We march in certainty, like the smart machines. We learn to sacrifice our freedom to collective knowledge imposed by others and for the sake of their guaranteed outcomes. This is the signature of the third modernity offered up by surveillance capital as its answer to our quest for effective life together.

The blunt behaviorist delivers his final truth: “A person does not act upon the world, the world acts upon him.”70

Social media marks a new era in the intensity, density, and pervasiveness of social comparison processes, especially for the youngest among us, who are “almost constantly online” at a time of life when one’s own identity, voice, and moral agency are a work in progress. In fact, the psychological tsunami of social comparison triggered by the social media experience is considered unprecedented. If television created more life dissatisfaction, what happens in the infinite spaces of social media?

Research has identified FOMO with compulsive Facebook use: FOMO sufferers obsessively checked their Facebook feeds—during meals, while driving, immediately upon waking or before sleeping, and so on. This compulsive behavior is intended to produce relief in the form of social reassurance, but it predictably breeds more anxiety and more searching.52

A three-phase investigation in 2014 found that spending a lot of time browsing profiles on Facebook produced a negative mood immediately afterward. Then, upon reflection, those users felt worse, reckoning that they had wasted their time. Instead of walking away, they typically chose to spend even more time browsing the network in the hope of feeling better, chasing the dream of a sudden and magical reversal of fortune that would justify past suffering. This cycle not only leads to more social comparison and more envy, but it can also predict depressive symptoms.56

Facebook is the crucible of this new dark science. It aims to perfect the relentless stimulation of social comparison in which natural empathy is manipulated and instrumentalized to modify behavior toward others’ ends. This synthetic hive is a devilish pact for a young person. In terms of sheer everyday effectiveness—contact, logistics, transactions, communications—turn away, and you are lost. And if you simply crave the fusion juice that is proof of life at a certain age and stage—turn away, and you are extinguished.

(3) the late twentieth century’s psychologically autonomous individual whose inner resources and capacity for moral judgment rise to the challenges of self-authorship that history demands and act as a bulwark against the predations of power. The self-authorship toward which young people strive carries forward these histories, strengthening, protecting, and rejuvenating each era’s claims to the sanctity and sovereignty of the individual person. What we have seen in Facebook is a living example of the third modernity that instrumentarianism proffers, defined by a new collectivism owned and operated by surveillance capital. The God view drives the computations. The computations enable tuning. Tuning replaces private governance and public politics, without which individuality is merely vestigial. And just as the uncontract bypasses social mistrust rather than healing it, the post-political societal processes that bind the hive rely on social comparison and social pressure for their durability and predictive certainty, eliminating the need for trust. Rights to the future tense, their expression in the will to will, and their sanctification in promises are drawn into the fortress of surveillance capital. On the strength of that expropriation, the tuners tighten their grasp, and the system flourishes. Industrial capitalism depended upon the exploitation and control of nature, with catastrophic consequences that we only now recognize. Surveillance capitalism, I have suggested, dep...

In the closing lines of Jean-Paul Sartre’s existential drama No Exit, the character Garcin arrives at his famous realization, “Hell is other people.” This was not intended as a statement of misanthropy but rather a recognition that the self-other balance can never be adequately struck as long as the “others” are constantly “watching.”

Surveillance capitalism’s antidemocratic and antiegalitarian juggernaut is best described as a market-driven coup from above. It is not a coup d’état in the classic sense but rather a coup de gens: an overthrow of the people concealed as the technological Trojan horse that is Big Other. On the strength of its annexation of human experience, this coup achieves exclusive concentrations of knowledge and power that sustain privileged influence over the division of learning in society: the privatization of the central principle of social ordering in the twenty-first century. Like the adelantados and their silent incantations of the Requirimiento, surveillance capitalism operates in the declarative form and imposes the social relations of a premodern absolutist authority. It is a form of tyranny that feeds on people but is not of the people. In a surreal paradox, this coup is celebrated as “personalization,” although it defiles, ignores, overrides, and displaces everything about you and me that is personal.

Technological “inevitability” is the mantra on which we are trained, but it is an existential narcotic prescribed to induce resignation: a snuff dream of the spirit.

This “seventh extinction” will not be of nature but of what has been held most precious in human nature: the will to will, the sanctity of the individual, the ties of intimacy, the sociality that binds us together in promises, and the trust they breed. The dying off of this human future will be just as unintended as any other.