Calling Bullshit: The Art of Skepticism in a Data-Driven World

by Carl T. Bergstrom

New-school bullshit can be particularly effective because many of us don’t feel qualified to challenge information that is presented in quantitative form. That is exactly what new-school bullshitters are counting on. To fight back, one must learn when and how to question such statements.

We have civic motives for wanting to help people spot and refute bullshit. It is not a matter of left- or right-wing ideology; people on both sides of the aisle have proven themselves skilled at creating and spreading misinformation. Rather (at the risk of grandiosity), we believe that adequate bullshit detection is essential for the survival of liberal democracy. Democracy has always relied on a critically thinking electorate, but never has this been more important than in the current age of fake news and international interference in the electoral process via propaganda disseminated over social media. In a December 2016 New York Times op-ed, Mark Galeotti summarized the best defense against this form of information warfare: Instead of trying to combat each leak directly, the United States government should teach the public to tell when they are being manipulated. Via schools and nongovernmental organizations and public service campaigns, Americans should be taught the basic skills necessary to be savvy media consumers, from how to fact-check news articles to how pictures can lie. As academics with decades of experience teaching data science, statistics, and related subjects at a public university, we know how to teach this sort of thinking. We believe that it can be done without taking sides, politically. You may not agree with us about the optimal size of the federal government, about what constitutes an acceptable degree of government involvement in our private lives, ...

Researchers who study animal behavior have been hard-pressed to demonstrate that any nonhuman animals have a theory of mind. But recent studies suggest that ravens may be an exception. When caching treats, they do think about what other ravens know. And not only do ravens act to deceive other birds sitting right there in front of them; they recognize that there might be other birds out there, unseen, who can be deceived as well.*1 That is pretty close to what we do when we bullshit on the Internet. We don’t see anyone out there, but we hope and expect that our words will reach an audience.

So why is there bullshit everywhere? Part of the answer is that everyone, crustacean or raven or fellow human being, is trying to sell you something. Another part is that humans possess the cognitive tools to figure out what kinds of bullshit will be effective. A third part is that our complex language allows us to produce an infinite variety of bullshit.

If I deliberately lead you to draw the wrong conclusions by saying things that are technically not untrue, I am paltering. Perhaps the classic example in recent history is Bill Clinton’s famous claim to Jim Lehrer on Newshour that “there is no sexual relationship [with Monica Lewinsky].” When further details came to light, Clinton’s defense was that his statement was true: He used the present-tense verb “is,” indicating no ongoing relationship. Sure, there had been one, but his original statement hadn’t addressed that issue one way or the other.

Usually when we get caught paltering, we are not forced to say anything as absurdly lawyerly as Bill Clinton’s “It depends upon what the meaning of the word ‘is’ is.”

Within linguistics, this notion of implied meaning falls under the area of pragmatics. Philosopher of language H. P. Grice coined the term implicature to describe what a sentence is being used to mean, rather than what it means literally. Implicature allows us to communicate efficiently.

Implicature provides a huge amount of wiggle room for people to say misleading things and then claim innocence afterward.

People all too often use this gulf between literal meaning and implicature to bullshit.

An important genre of bullshit known as weasel wording uses the gap between literal meaning and implicature to avoid taking responsibility for things.

When animals communicate, they typically send self-regarding signals. Self-regarding signals refer to the signaler itself rather than to something in the external world.

Perhaps the most important principle in bullshit studies is Brandolini’s principle. Coined by Italian software engineer Alberto Brandolini in 2014, it states: “The amount of energy needed to refute bullshit is an order of magnitude bigger than [that needed] to produce it.” Producing bullshit is a lot less work than cleaning it up. It is also a lot simpler and cheaper to do. A few years before Brandolini formulated his principle, Italian blogger Uriel Fanelli had already noted that, loosely translated, “an idiot can create more bullshit than you could ever hope to refute.”

So here we have a hypothesis that has been as thoroughly discredited as anything in the scientific literature. It causes serious harm to public health. And yet it will not go away. Why has it been so hard to debunk the rumors of a connection between vaccines and autism? This is Brandolini’s principle at work. Researchers have to invest vastly more time to debunk Wakefield’s arguments than he did to produce them in the first place.

Truth tellers face another disadvantage: The ways that we acquire and share information are changing rapidly. In seventy-five years we’ve gone from newspapers to news feeds, from Face the Nation to Facebook, from fireside chats to firing off tweets at four in the morning. These changes provide fertilizer for the rapid proliferation of distractions, misinformation, bullshit, and fake news. In the next chapter, we look at how and why this happened.

Roughly five hundred years after Filippo de Strata sounded the alarm about the printing press, sociologist Neil Postman echoed his sentiments: The invention of new and various kinds of communication has given a voice and an audience to many people whose opinions would otherwise not be solicited, and who, in fact, have little else but verbal excrement to contribute to public issues. If we wanted to condemn blogs, Internet forums, and social media platforms, we could scarcely say it better. But Postman was not referring to social media or even the Internet. He delivered this line a half century ago. In a 1969 lecture, he lamented the lowbrow programming on television, the vacuous articles in newspapers and magazines, and the general inanity of mass media. This kind of infotainment, he maintained, distracts consumers from the information that does matter—and distraction can itself be a form of disinformation. If religion is the opiate of the masses, Jersey Shore and Temptation Island are the spray canisters from which the masses huff metallic paint fumes. Since Postman’s lecture, we’ve undergone another revolution. The Internet has changed the way we produce, share, and consume information. It has altered the way we do research, learn about current events, interact with our peers, entertain ourselves, and even think. But why has the Internet also triggered a bullshit pandemic of unprecedented proportions?

Entrepreneur Steve Rayson looked at 100 million articles published in 2017 to determine what phrases were common in the headlines of articles that were widely shared. Their results will make you gasp in surprise—unless you’ve spent a few minutes on the Internet at some point in the past few years. The study found that the most successful headlines don’t convey facts, they promise you an emotional experience. The most common phrase among successful Facebook headlines, by nearly twofold, is “will make you,” as in “will break your heart,” “will make you fall in love,” “will make you look twice,” or “will make you gasp in surprise” as above. This phrase is also highly successful on Twitter. Other top phrases include “make you cry,” “give you goosebumps,” and “melt your heart.” Intellectual experiences cannot compete. Pause for a moment and think about what a huge shift this represents. Can you imagine The New York Times or your local newspaper with headlines that told you how you’d feel about each story, but not what the story actually entailed?

Much as the invention of the printing press allowed for a more diverse array of books, the advent of cable television allowed people to select specialized media outlets that closely reflected their views. Prior to 1987, the Fairness Doctrine of the US Federal Communications Commission (FCC) strived to ensure balanced coverage of controversial issues in news programming. But it was repealed under President Ronald Reagan. Hastened by the advent of the 24-hour news cycle, cable news channels proliferated and specialized in delivering specific political perspectives.

Social media is also fertile ground for disinformation, falsehoods that are spread deliberately. A study in 2018 found that about 2.6 percent of US news articles were false. This might not seem like a big percentage, but if every American read one article per day, it would mean that nearly eight million people a day were reading a false story.

It is not only new Internet users who are fooled. In December 2016, a website called AWD News published a frightening headline: “Israeli Defense Minister: If Pakistan Send Ground Troops to Syria on Any Pretext, We Will Destroy This Country with a Nuclear Attack.” The story contained several cues that should have tipped off a careful reader. The headline contained grammatical errors (“send” instead of “sends”). The story named the wrong person as the Israeli defense minister.*3 The article sat next to other implausible headlines such as “Clinton Staging Military Coup against Trump.” But it still fooled the person you would least want it to: Khawaja Muhammad Asif, defense minister of Pakistan. Asif responded with a threat of his own via Twitter: “Israeli def min threatens nuclear retaliation presuming pak role in Syria against Daesh…Israel forgets Pakistan is a Nuclear state too.” A single fake news piece led one major power to threaten another with a nuclear attack. It is one thing to mislead the infinitely gullible about Taylor Swift’s latest romance. It is another to tip the globe toward nuclear war.

Disinformation flows through a network of trusted contacts instead of being injected from outside into a skeptical society.

In 2016, chess grand master Garry Kasparov summarized this approach in a post on Twitter: “The point of modern propaganda isn’t only to misinform or push an agenda. It is to exhaust your critical thinking, to annihilate truth.”

In an Internet-connected world, governments have to worry about a new kind of counterfeiting—not of money, but of people. Researchers estimate that about half of the traffic on the Internet is due not to humans, but rather “bots,” automated computer programs designed to simulate humans. The scale of the problem is staggering. By 2018, Facebook had over two billion legitimate users—but in the same year deleted even more fake accounts: nearly three billion. Some bots act as information providers, pushing out their messages, usually for advertising purposes but occasionally in service of propaganda aims. Others emulate information consumers. “Click farms” use massive banks of cell phones to generate webpage views or YouTube video views for a fee.

Until recently, creating fake personalities on the Internet was tricky because in a social media world, we want to see the pictures of the people we follow. Without a picture, a user account on the Internet could be anyone. The computer security professional posting to a tech forum could be a kid in his mom’s basement. A fourteen-year-old girl in a chat room could be an undercover cop. The oil heiress in your in-box is undoubtedly a scam artist. But if we can see a picture, we tend to be less suspicious. Fake accounts sometimes use stock photos or images scraped from the Internet—but these were easily tracked down by savvy users using tools such as Google’s reverse image search. No longer. A new class of algorithms, collectively known as adversarial machine learning, can fashion photorealistic faces of nonexistent people out of whole cloth. The fabricated images are stunningly good. This is a dangerous period for a technology: It is widely available but few people know it’s being used. To raise public awareness, we developed a website called WhichFaceIsReal.com. A real photograph of a real person is paired with a computer-generated image of someone who does not exist. Your aim is to guess which is which. More than a million people have played the game on our website and the results show just how good the fakes are. People do not do much better than chance when they start playing, and even with lots of practice people still are fooled one time in five.

Director and comedian Jordan Peele created a public service announcement about fake news using this technology. Peele’s video depicts Barack Obama addressing the American people about fake news, misinformation, and the need for trusted news sources. Midway through the video, however, the face of Jordan Peele appears next to Obama, speaking the same words in perfect time, clearly the model from which Obama’s facial movements and expressions have been derived. Obama concludes, in his own voice but with Peele’s words: “How we move forward in the age of information is going to be the difference between whether we survive or whether we become some kind of fucked-up dystopia.”

There are three basic approaches for protecting ourselves against misinformation and disinformation online. The first is technology.

A second approach is governmental regulation.

A third and most powerful approach is education. If we do a good job of educating people in media literacy and critical thinking, the problem of misinformation and disinformation can be solved from the bottom up. That is our focus in this book, and in much of our professional lives.

If the data that go into the analysis are flawed, the specific technical details of the analysis don’t matter.

This objection introduces us to a pervasive source of bullshit. People take evidence about the association between two things, and try to sell you a story about how one causes the other. Circumcision is associated with autism. Constipation is associated with Parkinson’s disease. The marriage rate is associated with the suicide rate. But this doesn’t mean that circumcision causes autism, nor that constipation causes Parkinson’s, nor that marriage causes suicide. It is human nature to infer that when two things are associated, one causes the other. After all, we have evolved to find patterns in the world. Doing so helps us avoid danger, obtain food, deal with social interactions, and so much more. But often we are too quick to leap to conclusions about what causes what. In this chapter, we will show you how to think rigorously about associations, correlations, and causes—and how to spot bullshit claims that confuse one for the other.

Ask a philosopher what causation is, and you open an enormous can of worms. When a perfectly struck cue ball knocks the eight ball into the corner pocket, why do we say that the cue ball causes the eight ball to travel across the table and drop? The dirty secret is that although we all have an everyday sense of what it means for one thing to cause another, and despite endless debate in the fields of physics and metaphysics alike, there is little agreement on what causation is. Fortunately, we don’t need to know in order to use the notion of causation. In practice, we are usually interested in causation for instrumental purposes. We want to know how to cause things. We want to know why things went wrong in the past, so that we can make them go right in the future.

It is a truism that correlation does not imply causation. Do not leap carelessly from data showing the former to assumptions about the latter.*4

Correlation doesn’t imply causation—but apparently it doesn’t sell newspapers either.

One of the hallmark discoveries of social psychology is the role that delayed gratification plays in a successful life. At the heart of delayed gratification theory is an experiment known as the marshmallow test. A four-year-old is presented with alternative rewards: one marshmallow or two marshmallows. He is told that he can have a single marshmallow anytime—but if he can wait for a while, he can have two marshmallows. The experimenter then leaves the room and measures the amount of time until the child says screw it, and takes the single marshmallow. (After fifteen minutes of open-ended waiting, a child who has not yet given up receives the two-marshmallow reward. But seriously—do you remember how long fifteen minutes seemed at that age?) A number of studies have shown that children who can wait longer at age four have higher SAT scores in high school, and are rated by their parents as better-adjusted during adolescence. The authors of the original studies were careful to explain that their results demonstrated correlation: Delayed gratification is predictive of later academic success and emotional well-being. They did not demonstrate causation: The ability to delay gratification does not necessarily cause later success and well-being.*6 But as these results filtered through the popular press, the line between correlation and causation became blurred. The results of the marshmallow test and other related studies were reported as evidence that ability to delay gratificati...

But if one is not careful, looking at events chronologically can be misleading. Just because A happens before B does not mean that A causes B—even when A and B are associated. This mistake is so common and has been around for so long that it has a Latin name: post hoc ergo propter hoc. Translated, this means something like “after this, therefore because of it.”

Author Tyler Vigen has collected a delightful set of examples, and has a website where you can uncover these spurious correlations yourself.

There are many ways for error to creep into facts and figures that seem entirely straightforward. Quantities can be miscounted. Small samples can fail to accurately reflect the properties of the whole population. Procedures used to infer quantities from other information can be faulty. And then, of course, numbers can be total bullshit, fabricated out of whole cloth in an effort to confer credibility on an otherwise flimsy argument. We need to keep all of these things in mind when we look at quantitative claims. They say the data never lie—but we need to remember that the data often mislead.

To tell an honest story, it is not enough for numbers to be correct. They need to be placed in an appropriate context so that a reader or listener can properly interpret them. One thing that people often overlook is that presenting the numbers by themselves doesn’t mean that the numbers have been separated from any context. The choices one makes about how to represent a numerical value sets a context for that value.

The twelfth chapter of Carl Sagan’s 1996 book, The Demon-Haunted World, is called “The Fine Art of Baloney Detection.” In that chapter, Sagan rips into the advertising world for bombarding us with dazzling but irrelevant facts and figures. Sagan highlights the same problem we address in this chapter: people are easily dazzled by numbers, and advertisers have known for decades how to use numbers to persuade. “You’re not supposed to ask,” Sagan writes. “Don’t think. Buy.”

A notorious Breitbart headline similarly denied readers the opportunity to make meaningful comparisons. This particular bit of scaremongering proclaimed that 2,139 of the DREAM Act recipients—undocumented adults who came to the United States as children—had been convicted or accused of crimes against Americans.*3 That sounds like a big number, a scary number. But of course, the DREAM Act pertains to a huge number of people—nearly 700,000 held DACA status concurrently and nearly 800,000 were granted DACA status at some point before the program was eliminated. This means that only about 0.3 percent of all DACA recipients—fewer than 1 in 300—have been accused of crimes against Americans. That sounds better, but how does this number compare to similar rates for American citizens? With 0.75 percent of Americans behind bars, US citizens are twice as likely to be presently incarcerated as DACA recipients are to have been accused of a crime. About 8.6 percent of American citizens have been convicted of a felony at some point in their lives, making the DACA population look better still. Of course, DACA recipients are younger and had generally not been convicted of a crime prior to being awarded DACA status,*4 so they have had less time to commit crimes than your average American. But it turns out that 30 percent of Americans have been arrested for something other than a traffic violation by age twenty-three. Even assuming that Breitbart’s figures were correct, the news outlet prese...

Reporting numbers as percentages can obscure important changes in net values. For example, the US incarcerates African Americans at a shockingly high rate relative to members of other groups. African Americans are more than five times as likely as whites to be in jail or prison. In the year 2000, African Americans composed 12.9 percent of the US population but a staggering 41.3 percent of the inmates in US jails. Given that, it would seem to be good news that between the year 2000 and the year 2005, the fraction of the jail population that was African American declined from 41.3 percent to 38.9 percent. But the real story isn’t as encouraging as the percentage figures seem to indicate. Over this period, the number of African Americans in US jails actually increased by more than 13 percent. But this increase is obscured by the fact that over the same period, the number of Caucasians in US jails increased by an even larger fraction: 27 percent. This is an example of a more general issue with comparisons that involve fractions or percentages: Changing denominators obscure changes in numerators. The numerator—the top part of the fraction, here the number of jailed African Americans—increased substantially from 2000 to 2005. But the denominator—the bottom part of the fraction, here the total number of jailed Americans—increased by an even larger proportion over the same period. As a result, African Americans made up a smaller fraction of the incarcerated population in 2005 than...

Take college rankings such as the one compiled by U.S. News & World Report. This survey accounts for numerous aspects of colleges, including the acceptance rates for applicants and the average SAT scores of the incoming classes. Once these college rankings began to influence the number of applicants, admissions departments started to play the angles. To lower their acceptance rates and thereby appear more selective, some schools aggressively recruited applications, even from students unlikely to be admitted. Many schools switched to the common application so that candidates could apply by simply checking a box. To get credit for teaching small classes, some universities capped many class sizes at eighteen or nineteen—just below the U.S. News & World Report class-size cutoff of twenty. To lift their average SAT scores, schools employed a range of stratagems: not requiring SAT for international applicants who typically have lower scores, bringing in lower-scoring students in the spring semester when their scores are not counted, or even paying admitted students to retake the SAT, with bonuses for substantive increases in score. These efforts to game the system undercut the value of the rankings. The rankings ended up being influenced as much by admissions departments’ willingness to chase metrics as they did by the quality of schools’ applicants.

This problem is canonized in a principle known as Goodhart’s law. While Goodhart’s original formulation is a bit opaque,*8 anthropologist Marilyn Strathern rephrased it clearly and concisely: When a measure becomes a target, it ceases to be a good measure. In other words, if sufficient rewards are attached to some measure, people will find ways to increase their scores one way or another, and in doing so will undercut the value of the measure for assessing what it was originally designed to assess.

At around the same time that Goodhart proposed his law, psychologist Donald Campbell independently proposed an analogous principle: The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. Campbell illustrated his principle with the case of standardized testing in education: Achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.

MATHINESS Online voters selected truthiness as the word of the year in a 2006 survey run by the publishers of the Merriam-Webster dictionary. The term, coined in 2005 by comedian Stephen Colbert, is defined as “the quality of seeming to be true according to one’s intuition, opinion, or perception without regard to logic, factual evidence, or the like.” In its disregard for actual logic and fact, this hews pretty closely to our definition of bullshit. We propose an analogous expression, mathiness. Mathiness refers to formulas and expressions that may look and feel like math—even as they disregard the logical coherence and formal rigor of actual mathematics.

The so-called Trust Equation, laid out in a 2005 book of the same title, provides us with another illustration of mathiness. According to the Trust Equation,

Perhaps we aren’t supposed to take this equation-building approach literally. Why can’t we use these equations as a metaphor or a mode of expression? Mathematical equations are useful precisely for their precision of expression—but all of these examples fail to deliver this precision. You make a false promise when you suggest that happiness is an arithmetic sum of three things, if all you mean is that it increases with each of them. It is like promising chia-seed-and-spelt waffles sweetened with wildflower honey; topped with blackberry compote, a lingonberry reduction, and house-whipped sweet cream—but delivering an Eggo with artificial maple syrup. You can claim your description wasn’t meant to be taken literally, but why not say what you mean in the first place? We do not know why so many writers are prone to invent equations that layer a veneer of mathiness atop their qualitative arguments. Obviously it impresses some of the people, some of the time, but why aren’t these authors embarrassed to be caught out by the rest of us? Perhaps without a clear understanding of everything a mathematical equation entails, they fail to see the scale at which their faux equations underdeliver.

In statistical analysis, we deal with this problem by investigating small samples of a larger group and using that information to make broader inferences. If we want to know how many eggs are laid by nesting bluebirds, we don’t have to look in every bluebird nest in the country. We can look at a few dozen nests and make a pretty good estimate from what we find. If we want to know how people are going to vote on an upcoming ballot measure, we don’t need to ask every registered voter what they are thinking; we can survey a sample of voters and use that information to predict the outcome of the election. The problem with this approach is that what you see depends on where you look. To draw valid conclusions, we have to be careful to ensure that the group we look at is a random sample of the population. People who shop at organic markets are more likely to have liberal political leanings; gun show attendees are more likely to be conservative. If we conduct a survey of voters at an organic grocery store—or at a gun show—we are likely to get a misleading impression of sentiments citywide.

We should stress that a sample does not need to be completely random in order to be useful. It just needs to be random with respect to whatever we are asking about. Suppose we take an election poll based on only those voters whose names appear in the first ten pages of the phone book. This is a highly nonrandom sample of people. But unless having a name that begins with the letter A somehow correlates with political preference, our sample is random with respect to the question we are asking: How are you going to vote in the upcoming election?

Then there is the issue of how broadly we can expect a study’s findings to apply. When can we extrapolate what we find from one population to other populations? One aim of social psychology is to uncover universals of human cognition, yet a vast majority of studies in social psychology are conducted on what Joe Henrich and colleagues have dubbed WEIRD populations: Western, Educated, Industrialized, Rich, and Democratic. Of these studies, most are conducted on the cheapest, most convenient population available: college students who have to serve as study subjects for course credit.

WHAT YOU SEE DEPENDS ON WHERE YOU LOOK If you study one group and assume that your results apply to other groups, this is extrapolation. If you think you are studying one group, but do not manage to obtain a representative sample of that group, this is a different problem. It is a problem so important in statistics that it has a special name: selection bias. Selection bias arises when the individuals that you sample for your study differ systematically from the population of individuals eligible for your study.