Dataclysm: Love, Sex, Race, and Identity--What Our Online Lives Tell Us about Our Offline Selves

by Christian Rudder

this asymmetry is ending; the small noise, the crackle and hiss of the rest of us, is finally making it to tape. As the Internet has democratized journalism, photography, pornography, charity, comedy, and so many other courses of personal endeavor, it will, I hope, eventually democratize our fundamental narrative.

I’ve tried to capture all this with my mash-up title. Kataklysmos is Greek for the Old Testament Flood; that’s how the word “cataclysm” came to English.

A new parent is perhaps most sensitive to the milestones of getting older. It’s almost all you talk about with other people, and you get actual metrics at the doctor’s every few months.

Even now, in certain situations, we can find an excellent proxy, a sort of flash-forward to the possibilities. We can take groups of people at different points in their lives, compare them, and get a rough draft of life’s arc. This approach won’t work with music tastes, for example, because music itself also evolves through time, so the analysis has no control.

As usual, the good stuff lies in the distance between thought and action, and I’ll show you how we find it.

“To me, bad taste is what entertainment is all about. If someone vomits while watching one of my films, it’s like getting a standing ovation.”

Nostalgia used to be called mal du Suisse—the Swiss sickness.

The Internet has many regrettable sides to it, but that’s one thing that’s always stood it in good stead with me: it’s a writer’s world. Your life online is mediated through words. You work, you socialize, you flirt, all by typing. I honestly feel there’s a certain epistolary, Austenian grandness to the whole enterprise. No matter what words we use or how we tap out the letters, we’re writing to one another more than ever. Even if sometimes dam gerl is all we have to say.

There will be more words written on Twitter in the next two years than contained in all books ever printed.

Twitter gives us a sense of words not only as the building blocks of thought but as a social connector, which indeed has been the purpose of language since humanity hunched its way across the Serengeti.

In 1735, Leonhard Euler, as geniuses will do, came along and reduced what had been a colloquial question of neighborhoods and footpaths to an abstraction of dots and lines (formally: nodes and edges), and in doing so, he proved with rigor that the legend was true. He expressed the town as a network, and a discipline was founded.

Euler’s insight was that because you’re only supposed to cross each bridge once, to enter a new neighborhood you need a pair of bridges—one to get you in, another to get you out. So the solution is as simple as looking at the network plot and asking whether each point along your path, other than your beginning and end, has an even number of lines (a pair of bridges) attached.

Forty years ago, Stanley Milgram was mailing out parcels (kits with instructions and postage-paid envelopes) to a hundred people in Omaha, working on his “six degrees of separation,” hoping maybe a few dozen adventuresome souls would participate. His quaint methods—ingenious though they were—would give him the famous theory, but not quite its proof.

Early on, the best predictor of a relationship doesn’t depend on the couple’s social graph at all; for the first year or so of dating, the optimal method is how often they view each other’s profile. Only over time, as the page views go down and their mutual network fills out, does assimilation come to dominate the calculus. In other words, the curiosity, discovery, and (visual) stimulation of falling for someone is eventually replaced by the graph-theory equivalent of nesting.

As Steve Jobs said, “People don’t know what they want until you show it to them.” What he didn’t say is that showing them, especially in tech, means playing a game of Pin the Tail on the Donkey with several million people shouting advice.

For one button, the company couldn’t decide between two shades of blue, so they launched all forty-one shades in between to see which performed better. Know thyself: It was etched into a footstone of the Temple of Apollo at Delphi. But like the rest of the best wisdom that time has to offer, it goes right out the window as soon as anyone turns on a computer.

Before the advent of data like ours, one of the most quantified arenas in public life was sports. There you have real-time numbers on every conceivable interaction, and you have the data on an individual level, to be sliced and recombined at will.

Psychologists have a name for the interior patterns of belief that help a person organize information as he encounters it: schema. And our schema is still out of step with how most of us know the world should be.

Mankind is tribes within tribes. Or, putting it more beautifully, like the Korean proverb: “Over the mountains, mountains.” That’s the ruggedness of their peninsula and the endless difficulty of our fractured human terrain.

often the ugliest, most divisive, attitudes remain behind a veil of ego and cultural norms that is almost impossible to draw back, at least through direct questioning.

Google has become a repository for humanity’s collective id. It hears our confessions, our concerns, our secrets. It’s doctor, priest, psychiatrist, confidante, and above all, Google doesn’t have to ask us for a thing, because the question is always implied in the blank space of the interface: Hey, what’s on your mind?

All of which ailments, I have to point out, are probably the result of sitting at a computer for too damn long.

Historically, a presidential candidate can expect a modest boost, about 2 percentage points, in the popular vote in his home state.

A People’s History of the United States

The stoning metaphor comes up again and again when you read the commentary on episodes like these. It’s no coincidence that it’s the death penalty of choice for the ancient religions: there is no single executioner; the community carries out the punishment. No one can say who struck the fatal blow, because everyone did together.2 For a burgeoning tribe, fighting to preserve itself and its god in a hostile world, what better prescription could there be? There is strength in collective guilt, and guilt is diffused in the sharing. Extirpate the Other and make yourselves whole again.

We may think of human sacrifice as something from a savage past, and the physical act might now only exist in films about temples and doom, but the instinct remains within us, seemingly burned by deep time into the reaches of the animal mind.

Rumors are mentioned in our earliest texts. The archaic pantheons—Norse, Egyptian, Greek—all have a god dedicated to the dark art of gossip. The book of Proverbs treats the topic thoroughly; one verse from many cautions that “a man who lacks judgment derides his neighbor, but a man of understanding holds his tongue.” “Judge not lest you be judged” is one of the most famous phrases in the whole Bible.

The theory is, when ancient man had to figure out if x was true, language gave him a way to investigate. So he talked about it. And, true or false, word spread. Rumors—essentially group speculation over the truth of an idea—became a way to build bonds and social capital. Stories create status for those who share them, especially when they concern important individuals, because information about powerful people is a form of power itself.

Be the first to spread the news, get more retweets. Say something especially cutting, and your followers applaud your wit. The social capital you build by sharing information is now explicit; in fact, it’s in little numbers that increment before your very eyes.

“To the extent people do have an agenda in spreading rumors it’s directed more at the people they’re spreading them to, rather than at the subject of the rumor.” The Internet gives people a wider audience than ever before.

social media empowers you to the extent that it makes you worth tearing down. At the same time, it gives everyone else the tools to do it. Demon Rumor now has a million mouths.

As I pointed out earlier, by 2015, Twitter users will have exchanged more words than have ever been printed. The question is how to harness the chatter.

Mathematical models already exist to predict the outcome of armed conflict—how long it will last, who will win, and how many people will die—and the models of late have learned to accommodate guerrilla warfare, since that’s the shape of today’s war.

In any event, the software first establishes a group’s central personalities by looking at its social graph—much like we portrayed a marriage as edges and nodes before, the software lays out the network, then algorithmically determines its most important dots. Next, it looks at what those dots are saying. Condor has found that while the foci of a movement are positive in their word choice, the movement is vibrant. But negative words like “hate,” “not,” “lame,” and “never” signal decline, and when, as The Economist put it, “complaints about idiots in one’s own movement or such infelicities as the theft of beer by a fellow demonstrator” begin to appear, the movement is all but over. Oh, Occupy!

Any software that follows the thread of a thought through a network must track not only the idea but the “susceptibility” of people exposed to it.

It shows we fight hardest against those who can least fight back. And, above all, it runs to ground our age-old desire to raise ourselves up by putting other people down. Scientists have established that the drive is as old as time, but this doesn’t mean they understand it yet. As Gandhi put it, “It has always been a mystery to me how men can feel themselves honored by the humiliation of their fellow beings.”

In social science, knowledge, like water, often takes the shape of its vessel. So if we want to take all the self-statements I’ve collected and pull from them a sense of who the writers are—what makes ethnicities and sexes and orientations unique—we’ll need to develop an algorithm that takes the “us” out of it and leaves just the “them.”

Zipf’s law, an observed statistical property of language that, like so much of the best math, lies somewhere between miracle and coincidence.1 It states that in any large body of text, a word’s popularity (its place in the lexicon, with 1 being the highest ranking) multiplied by the number of times it shows up, is the same for every word in the text. Or, very elegantly: rank × number = constant This law holds for the Bible, the collected lyrics of ’60s pop songs, and the canonical corpus of English literature (the Oxford English Corpus),

And as further evidence of its deep connection with the human experience, Zipf’s law also describes a wide variety of our social constructs: the sizes of cities, for example, and income distribution across a population. What it means for our purpose here is that because most of language is just a small body of repeated patterns, the use of a word drops off rapidly.

Now that we understand how rankings and usage frequency compare, the next step is to use those rankings to our advantage.

Because every data set has its quirks, researchers must often build tools from scratch, as we have here. Whenever you do this, it’s good to check your method against some familiar outcomes. Imagine a shipwright with a new boat: who knows what’ll happen once it’s out on the open ocean—so best to check for holes close to shore.

I will point out a few broad trends: white people differentiate themselves mostly by their hair and eyes, Asians by their country of origin, Latinos by their music.

From empathy and sexuality to science inclination and extroversion, statistical analysis of 122 different characteristics involving 13,301 individuals shows that men and women, by and large, do not fall into different groups.

Cultural differences, even if they’re occasionally laughable, make the world a richer place.

The Mars/Venus thing, metaphor though it is, reminds me that the heavens are an ancient reference point for science. Aristotle looked to the emptiness overhead to verify his aether. Newton confirmed his law of inverse squares through the motion of Mars. Even Einstein wasn’t truly Einstein until the sun and moon said so, in a 1919 eclipse that confirmed the theory of General Relativity. Even though we’re working on nothing so grand as all that here, I have to say I hope that paper’s snarky strikeout typeface is premature, at least for the things we like and talk about and the ways we spend our time. Look at it this way: if there were no planet out there but Earth, it would be a very boring universe.

prejudice unchallenged is prejudice perpetuated.

String theory predicts that our physical existence requires somewhere between ten and twenty-six dimensions. Our emotional universe surely has that many and more. And in combining these spaces—our interior landscape with our external world—we can portray existence with a new depth.

Many of our own American states were created by royal charter or act of Congress, their borders drawn by people who would never see the land in person.

Since then, smartphones, each one with a tiny GPS pinging, have revolutionized cartography. Matthew Zook, a geographer at the University of Kentucky, has partnered with data scientists there to create what they call the DOLLY Project (Digital OnLine Life and You)—it’s a searchable repository of every geotagged tweet since December 2011, meaning Zook and his team have compiled billions of interrelated sentiments, each with a latitude and longitude attached. DOLLY is an incredibly versatile resource, the output of which is only now being explored.

A demonstration of DOLLY’s power on YouTube shows it tracking the Dutch holiday of Sint Maarten, a sort of Germanic Halloween where children go door to door singing for candy.