Calling Bullshit: The Art of Skepticism in a Data-Driven World

by Carl T. Bergstrom

THE WORLD IS AWASH WITH bullshit, and we’re drowning in it. Politicians are unconstrained by facts. Science is conducted by press release. Silicon Valley startups elevate bullshit to high art. Colleges and universities reward bullshit over analytic thought. The majority of administrative activity seems to be little more than a sophisticated exercise in the combinatorial reassembly of bullshit. Advertisers wink conspiratorially and invite us to join them in seeing through all the bullshit. We wink back—but in doing so drop our guard and fall for the second-order bullshit they are shoveling at us. Bullshit pollutes our world by misleading people about specific issues, and it undermines our ability to trust information in general. However modest, this book is our attempt to fight back.

For all of its successes, we feel that higher education in STEM disciplines—science, technology, engineering, and mathematics—has dropped the ball in this regard. We generally do a good job teaching mechanics: students learn how to manipulate matrices, transfect cells, run genomic scans, and implement machine learning algorithms. But this focus on facts and skills comes at the expense of training and practice in the art of critical thinking. In the humanities and the social sciences, students are taught to smash conflicting ideas up against one another and grapple with discordant arguments. In STEM fields, students seldom are given paradoxes that they need to resolve, conflicting forms of evidence that they must reconcile, or fallacious claims that they need to critique. As a result, college graduates tend to be well prepared to challenge verbal arguments and identify logical fallacies but surprisingly acquiescent in the face of quantitative claims. Of course, the same holds true for secondary education. If STEM education incorporated the interrogatory teaching practices already common in the humanities, schools could shape a generation of students prepared to call bullshit on statistical statements and artificial intelligence analyses just as comfortably as current students can on political, ethical, artistic, and philosophical claims.

Of all human institutions, science seems as though it ought to be free from bullshit—but it isn’t. We believe that public understanding of science is critical for an informed electorate, and we want to identify the many obstacles that interfere with that understanding.

We impose strong social sanctions on liars. If you get caught in a serious lie, you may lose a friend. You may get punched in the nose. You may get sued in a court of law. Perhaps worst of all, your duplicity may become the subject of gossip among your friends and acquaintances. You may find yourself no longer a trusted partner in friendship, love, or business. With all of these potential penalties, it’s often better to mislead without lying outright. This is called paltering.

Perhaps the most important principle in bullshit studies is Brandolini’s principle. Coined by Italian software engineer Alberto Brandolini in 2014, it states: “The amount of energy needed to refute bullshit is an order of magnitude bigger than [that needed] to produce it.” Producing bullshit is a lot less work than cleaning it up. It is also a lot simpler and cheaper to do.

Satirist Jonathan Swift wrote in 1710 that “falsehood flies, and truth comes limping after it.”*5

The full quotation is: “Falsehood flies, and truth comes limping after it, so that when men come to be undeceived, it is too late; the jest is over, and the tale hath had its effect: like a man, who hath thought of a good repartee when the discourse is changed, or the company parted; or like a physician, who hath found out an infallible medicine, after the patient is dead.”

IF IN 1990 YOU HAD told us that by 2020 nearly half of the people on the planet would carry a wallet-size machine that could instantly look up any fact in the world—a “smartphone”—we would have predicted an end to bullshit. How could you bullshit someone who could check your claims easily, immediately, and costlessly? Apparently people have neither the time nor the inclination to use smartphones this way. Instead, smartphones have become just one more vehicle for spreading bullshit. On the positive side, you can have a decent dinner conversation without being fact-checked thirty times. On the negative side, bullshit goes largely unchallenged. Technology didn’t eliminate our bullshit problem, it made the problem worse. In this chapter, we will explore how that happened. In short, the rise of the Internet changed what kinds of information get produced, how information is shared, and the ways in which we find the information that we want. While much good has come of the Internet revolution, there have been major drawbacks as well. Fluff and glitter have overtaken serious, in-depth, thoughtful content. News coverage has become increasingly partisan. Misinformation, disinformation, and fake news abound. We will consider these issues in turn.

Pity the soul who hopes to hold back a revolution in information technology.

Let’s begin by looking at what gets published. Through the 1980s, publishing required money—a lot of it. Typesetting was expensive, printing required substantial overhead, and distribution involved getting physical paper into the hands of readers. Today, anyone with a personal computer and an Internet connection can produce professional-looking documents and distribute them around the world without cost. And they can do so in their pajamas.

This democratization has a dark side as well. Aided by viral spread across social media, amateur writers can reach audiences as large as those of professional journalists. But the difference in the reporting quality can be immense. A typical Internet user lacks the journalistic training, let alone the incentives to report accurately. We can access more information than ever, but that information is less reliable.

Filippo de Strata feared that the works of Ovid might crowd out the Bible. We fear that the mindless lists, quizzes, memes, and celebrity gossip that proliferate on social media might crowd out thoughtful analyses of the sort you see in The New York Times or The Wall Street Journal. Every generation thinks that its successor’s lazy habits of mind will bring on a cultural and intellectual decline. It may be a stodgy lament that has been repeated for thousands of years, but it’s our turn now, and we’re not going to miss the opportunity to grumble.

A satirical website published a headline proclaiming that “70% of Facebook Users Only Read the Headline of Science Stories before Commenting.” The story began by noting that most people don’t read stories before sharing them on social media either. After a couple of sentences, the text gave way to paragraph after paragraph of the standard “lorem ipsum dolor…”—random text used as filler for webpage layouts. The post was shared tens of thousands of times on social media, and we don’t know how many of those who did so were in on the joke.

All of this fluff and glitter does more than just dumb down the national conversation: It opens the door for bullshit. The unvarnished truth is no longer good enough. Straight-up information cannot compete in this new marketplace.

Much as the invention of the printing press allowed for a more diverse array of books, the advent of cable television allowed people to select specialized media outlets that closely reflected their views. Prior to 1987, the Fairness Doctrine of the US Federal Communications Commission (FCC) strived to ensure balanced coverage of controversial issues in news programming. But it was repealed under President Ronald Reagan. Hastened by the advent of the 24-hour news cycle, cable news channels proliferated and specialized in delivering specific political perspectives.

Online, it’s the same story, only more so. Even mainstream outlets deliver news with a partisan slant. We find ourselves isolated in separate echo chambers. Publishers such as Breitbart News Network and The Other 98% go one step further, pushing what is known as hyperpartisan news. Their stories may be based in fact, but they are so strongly filtered through an ideological lens that they often include significant elements of untruth. Publishers churn out partisan and hyperpartisan content because it pays to do so. Social media favors highly partisan content. It is shared more than mainstream news, and once shared, it is more likely to be clicked on. Deepening the ideological divide has become a lucrative business. MIT professor Judith Donath has observed that even when people appear to be talking about other things, they’re often talking about themselves. Suppose I log on to Facebook and share a false—even absurd—story about how airplane contrails are endocrine-disrupting chemicals being sprayed as part of a liberal plot to lower the testosterone levels of America’s youth. I may not be as interested in having you believe my claims about contrails as I am in signaling my own political affiliations. Sharing an article like this signals that I belong to a group of people who believe in conspiracy theories and distrust the “liberal agenda” in America. And if that’s my aim, it doesn’t matter to me whether the story is true or false. I may not have read it, I may not care if you...

Blindfolded and submerged in a pool, I shout “Marco!” If I do so correctly, my network of acquaintances sends back an encouraging chorus. “Polo! Polo! Polo!” Participating on social media is only secondarily about sharing new information; it is primarily about maintaining and reinforcing common bonds. The danger is that, in the process, what was once a nationwide conversation fragments beyond repair. People begin to embrace tribal epistemologies in which the truth itself has less to do with facts and empirical observation than with who is speaking and the degree to which their message aligns with their community’s worldview.

Riffing on Allen Ginsberg, tech entrepreneur Jeff Hammerbacher complained in 2011 that “the best minds of my generation are thinking about how to make people click ads. That sucks.” The problem is not merely that these “best minds” could have been devoted to the artistic and scientific progress of humankind. The problem is that all of this intellectual firepower is devoted to hijacking our precious attention and wasting our minds as well. The Internet, social media, smartphones—we are exposed to increasingly sophisticated ways of diverting our attention. We become addicted to connectivity, to meaningless checking, to a life of fragmented attention across myriad streams of digital information. In short, the algorithms driving social media content are bullshitters. They don’t care about the messages they carry. They just want our attention and will tell us whatever works to capture it.

Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade or impress an audience by distracting, overwhelming, or intimidating them with a blatant disregard for truth, logical coherence, or what information is actually being conveyed.

While he has not written about bullshit directly, the sociologist of science Bruno Latour has had a formative effect on our thinking about how people bullshit their audiences. Latour looks at the power dynamics between an author and a reader. In Latour’s worldview, a primary objective of nonfiction authors is to appear authoritative. One good way to do this is to be correct, but that is neither necessary nor sufficient. Correct or not, an author can adopt a number of tactics to make her claims unassailable by her readers—who in turn strive not to be duped. For example, the author can line up a phalanx of allies by citing other writers who support her point, or whose work she builds upon. If you question me, she implies, you have to question all of us. She can also deploy sophisticated jargon. Jargon may facilitate technical communication within a field, but it also serves to exclude those who have not been initiated into the inner circle of a discipline.

But it doesn’t need to. The central theme of this book is that you usually don’t have to open the analytic black box in order to call bullshit on the claims that come out of it. Any black box used to generate bullshit has to take in data and spit results out, like the diagram on this page.

Most often, bullshit arises either because there are biases in the data that get fed into the black box, or because there are obvious problems with the results that come out. Occasionally the technical details of the black box matter, but in our experience such cases are uncommon. This is fortunate, because you don’t need a lot of technical expertise to spot problems with the data or results. You just need to think clearly and practice spotting the sort of thing that can go wrong. In the pages that follow, we will show you how to do precisely this.

If the data that go into the analysis are flawed, the specific technical details of the analysis don’t matter. One can obtain stupid results from bad data without any statistical trickery. And this is often how bullshit arguments are created, deliberately or otherwise. To catch this sort of bullshit, you don’t have to unpack the black box. All you have to do is think carefully about the data that went into the black box and the results that came out. Are the data unbiased, reasonable, and relevant to the problem at hand? Do the results pass basic plausibility checks? Do they support whatever conclusions are drawn?

Ask a philosopher what causation is, and you open an enormous can of worms. When a perfectly struck cue ball knocks the eight ball into the corner pocket, why do we say that the cue ball causes the eight ball to travel across the table and drop? The dirty secret is that although we all have an everyday sense of what it means for one thing to cause another, and despite endless debate in the fields of physics and metaphysics alike, there is little agreement on what causation is.

But it is rarely straightforward to figure out what effects an action will have. A large fraction of the time all we have to work with is information about correlations. Scientists have a number of techniques for measuring correlations and drawing inferences about causality from these correlations. But doing so is a tricky and sometimes contentious business, and these techniques are not always used as carefully as they ought to be. Moreover, when we read about recent studies in medicine or policy or any other area, these subtleties are often lost. It is a truism that correlation does not imply causation. Do not leap carelessly from data showing the former to assumptions about the latter.*4 This is difficult to avoid, because people use data to tell stories. The stories that draw us in show a clear connection between cause and effect. Unfortunately, one of the most frequent misuses of data, particularly in the popular press, is to suggest a cause-and-effect relationship based on correlation alone. This is classic bullshit, in the vein of our earlier definition, because often the reporters and editors responsible for such stories don’t care what you end up believing. When they tell you that drinking wine prevents heart disease, they are not trying to lead you into alcoholism or away from behaviors that promote cardiac health. At best, they are trying to tell a good story. At worst, they are trying to compel you to buy a magazine or click on a link.

One of the hallmark discoveries of social psychology is the role that delayed gratification plays in a successful life.

But here’s the thing. These prescriptions are unwarranted, because we don’t actually have strong evidence that the ability to delay gratification causes subsequent success. When a research team went back and attempted to replicate the original marshmallow studies with a larger sample size and additional controls, they found only a fraction of the original effect. Moreover, there was a single factor that seemed responsible both for a child’s ability to delay gratification and for success during adolescence: the parents’ socioeconomic status.*7

There is a key distinction between a probabilistic cause (A increases the chance of B in a causal manner), a sufficient cause (if A happens, B always happens), and a necessary cause (unless A happens, B can’t happen).

OUR WORLD IS THOROUGHLY QUANTIFIED. Everything is counted, measured, analyzed, and assessed. Internet companies track us around the Web and use algorithms to predict what we will buy. Smartphones count our steps, measure our calls, and trace our movements throughout the day. “Smart appliances” monitor how we use them and learn more about our daily routines than we might care to realize. Implanted medical devices collect a continuous stream of data from patients and watch for danger signs in real time. During service visits, our cars upload data about their performance and about our driving habits. Arrays of sensors and cameras laid out across our cities monitor everything from traffic to air quality to the identities of passersby. Instead of collecting data about what people do through costly studies and surveys, companies let people come to them—and then record what those consumers do. Facebook knows whom we know; Google knows what we want to know. Uber knows where we want to go; Amazon knows what we want to buy. Match knows whom we want to marry; Tinder knows whom we want to be swiped by. Data can help us understand the world based upon hard evidence, but hard numbers are a lot softer than one might think. It’s like the old joke: A mathematician, an engineer, and an accountant are applying for a job. They are led to the interview room and given a math quiz. The first problem is a warm-up: What is 2 + 2? The mathematician rolls her eyes, writes the numeral 4, and moves on...

In these examples, we are observing a population with a range of values—a range of heights, for instance—and then summarizing that information with a single number that we call a summary statistic. For example, when describing the tall Dutch, we reported a mean height. Summary statistics can be a nice way to condense information, but if you choose an inappropriate summary statistic you can easily mislead your audience. Politicians use this trick when they propose a tax cut that will save the richest 1 percent of the population hundreds of thousands of dollars but offer no tax reduction whatsoever to the rest of us. Taking a mean of the tax cuts, they report that their tax plan will save families an average $4,000 per year. Maybe so, but the average family—if by that we mean one in the middle of the income range—will save nothing. The majority of us would be better off knowing the tax cut for a family with the median income. The median is the “middle” income: Half of US families earn more and half earn less. In this particular case, the median family would get no tax cut at all, because the tax cut benefits only the top 1 percent.

To tell an honest story, it is not enough for numbers to be correct. They need to be placed in an appropriate context so that a reader or listener can properly interpret them. One thing that people often overlook is that presenting the numbers by themselves doesn’t mean that the numbers have been separated from any context. The choices one makes about how to represent a numerical value sets a context for that value. So what does it mean to tell an honest story? Numbers should be presented in ways that allow meaningful comparisons.

Carl Sagan’s 1996 book, The Demon-Haunted World,

Read later

Percentages can be particularly slippery when we use them to compare two quantities. We typically talk about percentage differences: “a 40 percent increase,” “22 percent less fat,” etc. But what is this a percentage of? The lower value? The higher value? This distinction matters.

Changing denominators wreak havoc on percentages.

When scientists measure the molecular weights of the elements, the elements do not conspire to make themselves heavier and connive to sneak down the periodic table. But when administrators measure the productivity of their employees, they cannot expect these people to stand by idly: Employees want to look good. As a result, every time you quantify performance or rank individuals, you risk altering the behaviors you are trying to measure.

Marilyn Strathern rephrased it clearly and concisely: When a measure becomes a target, it ceases to be a good measure. In other words, if sufficient rewards are attached to some measure, people will find ways to increase their scores one way or another, and in doing so will undercut the value of the measure for assessing what it was originally designed to assess. We see this in any number of domains. In the sciences, the use of citation metrics to measure journal quality has led editors to game the system. Some pressure authors to include citations to papers in the same journal. Some publish an excess of articles in January, when they have the most time to be cited during the year. Others publish annual summary papers that cite many of the articles published within a year; yet others shift their focus to disciplines or types of articles that tend to attract more citations. All of these perverse behaviors undermine the mission of the journals and the effectiveness of citation measures as indicators of quality. When the manager of an auto dealership provides bonuses to salespeople for hitting certain volume targets, the sales force will have incentives to offer larger discounts to their customers, in order to make the sale quickly. If salespeople were trying to maximize profits in making each sale, one might be able to use the number of cars sold to approximate how much money each salesperson brings in. But once the number of cars sold becomes a target, salespeople alter the...

Achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. If no one knew that you were going to evaluate schools by looking at test scores of students, test scores might provide a reasonable way of measuring schools’ effectiveness. But once teachers and administrators recognize that test scores will be used to evaluate their effectiveness, they have every incentive to find ways of raising their students’ scores—even at the expense of the quality of their education. They may “teach to the test,” for example, rather than teaching critical thinking. The consequences are twofold. Once this happens, test scores lose much of their value as a means of assessing a school’s performance. The process of measurement may even actively undermine the quality of education, as time and effort are shifted away from valuable activity and toward the kind of memorization that improves test scores but is nearly useless from an educational perspective. The same problem occurs in the sciences as well. A few years back, Jevin penned a commentary about the dangers of using quantitative metrics to measure the quality of scientific research. Getting the wrong answer is not the worst thing a metric can do. It’s much worse to provide peop...

Goodhart originally expressed his law as: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

Often we want to learn about the individuals in some group. We might want to know the incomes of families in Tucson, the strength of bolts from a particular factory in Detroit, or the health status of American high school teachers. As nice as it would be to be able to look at every single member of the group, doing so would be expensive if not outright infeasible. In statistical analysis, we deal with this problem by investigating small samples of a larger group and using that information to make broader inferences. If we want to know how many eggs are laid by nesting bluebirds, we don’t have to look in every bluebird nest in the country. We can look at a few dozen nests and make a pretty good estimate from what we find. If we want to know how people are going to vote on an upcoming ballot measure, we don’t need to ask every registered voter what they are thinking; we can survey a sample of voters and use that information to predict the outcome of the election. The problem with this approach is that what you see depends on where you look. To draw valid conclusions, we have to be careful to ensure that the group we look at is a random sample of the population. People who shop at organic markets are more likely to have liberal political leanings; gun show attendees are more likely to be conservative. If we conduct a survey of voters at an organic grocery store—or at a gun show—we are likely to get a misleading impression of sentiments citywide. We also need to think about wh...

We should stress that a sample does not need to be completely random in order to be useful. It just needs to be random with respect to whatever we are asking about. Suppose we take an election poll based on only those voters whose names appear in the first ten pages of the phone book. This is a highly nonrandom sample of people. But unless having a name that begins with the letter A somehow correlates with political preference, our sample is random with respect to the question we are asking: How are you going to vote in the upcoming election?*1 Then there is the issue of how broadly we can expect a study’s findings to apply. When can we extrapolate what we find from one population to other populations? One aim of social psychology is to uncover universals of human cognition, yet a vast majority of studies in social psychology are conducted on what Joe Henrich and colleagues have dubbed WEIRD populations: Western, Educated, Industrialized, Rich, and Democratic. Of these studies, most are conducted on the cheapest, most convenient population available: college students who have to serve as study subjects for course credit. How far can we generalize based on the results of such studies? If we find that American college students are more likely to engage in violent behavior after listening to certain kinds of music, we need to be cautious about extrapolating this result to American retirees or German college students, let alone people in developing countries or members of trad...

Our friend applied one of the most important rules for spotting bullshit: If something seems too good or too bad to be true, it probably is.

Selection effects appear everywhere, once you start looking for them. A psychiatrist friend of ours marveled at the asymmetry in how psychiatric disorders are manifested. “One in four Americans will suffer from excessive anxiety at some point,” he explained, “but in my entire career I have only seen one patient who suffered from too little anxiety.” Of course! No one walks into their shrink’s office and says “Doctor, you’ve got to help me. I lie awake night after night not worrying.” Most likely there are as many people with too little anxiety as there are with too much. It’s just that they don’t go in for treatment. Instead they end up in prison, or on Wall Street.

The same mathematical principles explain the curious fact that most likely, the majority of your friends have more friends than you do. This is not true merely because you are the kind of person who reads a book about bullshit for fun; it’s true of anyone, and it’s known as the friendship paradox.

Okay. Forget we mentioned that. Back to statistics. Selection effects like this one are sometimes known as observation selection effects because they are driven by an association between the very presence of the observer and the variable that the observer reports. In the class size example, if we ask students about the size of their classes, there is an association between the presence of the student observer and the class size. If instead we ask teachers about the sizes of their classes, there are no observation selection effects because each class has only one teacher—and therefore there is no association between the presence of a teacher in the classroom and the size of the class.

Finally, we want to note that Berkson’s paradox provides a hitherto unrecognized explanation for William Butler Yeats’s dark observation in his classic modernist poem “The Second Coming”: The best lack all conviction, while the worst Are full of passionate intensity.

Another problem is that there is no caveat about the right-censoring issue in the data graphic itself. In a social media environment, we have to be prepared for data graphics—at least those from pieces directed to a popular audience—to be shared without the accompanying text. In our view, these data never should have been plotted in the way that they were. The graph here tells a story that is not consistent with the conclusions that a careful analysis would suggest.

Wellness programs raise ethical questions about employers having this level of control and ownership over employees’ bodies. But there is also a fundamental question: Do they work? To answer that, we have to agree about what wellness programs are supposed to do. Employers say that they offer these programs because they care about their employees and want to improve their quality of life. That’s mostly bullshit. A sand volleyball court on the company lawn may have some recruiting benefits. But the primary rationale for implementing a wellness program is that by improving the health of its employees, a company can lower insurance costs, decrease absenteeism, and perhaps even reduce the rate at which employees leave the company. All of these elements contribute to the company’s bottom line.

Simply surveying people who are listed in the phone book can be a problem as well. Doing so excludes people who do not have landlines and people who have unlisted numbers. Moreover, those who take the time to answer a telephone survey may differ systematically from those who screen their calls or hang up on the interviewer. As such, any form of sampling from a phone book may substantially undercount certain demographic groups including younger people and some minorities. This specific issue has become a major problem for telephone polling. Pollsters increasingly use cell phone interviews, but this approach comes with its own suite of problems.

Contrary to what everyone assumes, the Florida Stand Your Ground graphic was not intended to mislead. It was just poorly designed. This highlights one of the principles for calling bullshit that we espouse. Never assume malice or mendacity when incompetence is a sufficient explanation, and never assume incompetence when a reasonable mistake can explain things.

Fortunately, the authors of this particular paper provide both views of the data so that we can see how misleading it can be to plot the means of binned data. But authors are not always so transparent. Sometimes only the binned means will appear in a scientific paper or a news story about research results. Be on the lookout, lest you be duped into thinking that a trend is much stronger than it actually is.