How We Know What Isn't So (A Psychological Study on Logic)

by Thomas Gilovich

There are a couple of points to be made about this issue. First, cognitive explanations are not inherently more parsimonious than motivational ones. They can, and do, involve as many assumptions as motivational accounts.

Given a model in which the motivational system is more fundamental, motivational explanations would be more parsimonious.

To the extent that there is a motivational “engine” responsible for our self-serving biases and beliefs, it is one that delivers its effects through processes that look suspiciously cognitive.

Essentially the same point has been articulated by social psychologist Ziva Kunda, who argues that people are indeed more likely to believe things they want to believe, but that their capacity to do so is constrained by objective evidence and by their ability “… to construct a justification of their desired conclusion that would persuade a dispassionate observer. They draw the desired conclusion only if they can muster up the evidence necessary to support it.”22 It is informative in this respect that people generally think of themselves as objective.*

When we prefer to believe something, we may approach the relevant evidence by asking ourselves, “what evidence is there to support this belief?” If we prefer to believe that a political assassination was not the work of a lone gunman, we may ask ourselves about the evidence that supports a conspiracy theory. Note that this question is not unbiased: It directs our attention to supportive evidence and away from information that might contradict the desired conclusion. Because it is almost always possible to uncover some supportive evidence, the asymmetrical way we frame the question makes us overly likely to become convinced of what we hope to be true.

By judiciously choosing the right people to consult, we can increase our chances of hearing what we want to hear. Smokers can discuss their habit’s health risks with other smokers; Nixon fans can explore the “real meaning” of the Watergate scandal with those of similar ideological bent. There are a number of physiologists at Cornell who differ in their assessments of the importance of dietary fat as a determinant of serum cholesterol and arteriosclerosis. This variability in expert opinion gives members of the Cornell community an opportunity to find support for whatever eating practices they wish. Those who need to justify the lost opportunities brought on by an austere diet can talk with someone willing and able to describe the latest studies testifying to the evils of dietary fat; those with an appetite for Continental cuisine can talk with someone eager to discuss the critical flaws of those very same studies. We seek opinions that are likely to support what we want to be true.

People’s preferences influence not only the kind of information they consider, but also the amount they examine.

For desired conclusions, in other words, it is as if we ask ourselves, “Can I believe this?”, but for unpalatable conclusions we ask, “Must I believe this?”

By basing our definitions of what constitutes being, say, athletic, intelligent, or generous on our own idiosyncratic strengths on these dimensions, almost all of us can think of ourselves as better than average and have some “objective” justification for doing so.

First, it has been shown that people are particularly inclined to think of themselves as above average on ambiguous traits—those for which the definition of what constitutes excellence can most readily be construed in self-serving ways.

The students were also asked to rate their own standing on these characteristics. As expected, the students tended to think that the characteristics at which they excelled were most important in determining what constitutes a successful college student.

We describe the formation of beliefs with numerous references to possession, as when we say that “I adopted the belief,” “he inherited the view,” “she acquired her conviction,” or, if a potential belief is rejected, “I don’t buy that.” When someone believes in something, we refer to fact that “she holds a belief,” or “he clings to his belief.” When a belief is “given up,” we state that “he lost his belief,” “she abandoned her convictions,” or “I disown my earlier stand.”

What ails the truth is that it is mainly uncomfortable, and often dull. The human mind seeks something more amusing, and more caressing.

More to the point, however, this chapter also examines how this need or desire to tell a good story can distort the accuracy of information we receive secondhand, and thus bias some of the most important information upon which we base our beliefs.

Reliability of stories as evidence.

Sharpening and Leveling. For the purpose of understanding the formation of erroneous beliefs, it is important to note that satisfying even these very basic enabling conditions can introduce distortion in what is communicated. Classic studies by psychologists F. C. Bartlett10 and Gordon Allport and Leo Postman11 demonstrate that when people are given a message to relay to someone else, they rarely convey the message verbatim. The limits of human memory and the implicit demand that the listener not be burdened with too many details constrain the amount and kind of information that is transmitted.

A particularly interesting consequence of the processes of sharpening and leveling concerns our impressions of people we only know about secondhand.12 Everyday experience seems to tell us that we often develop exaggerated or extreme impressions of people we have heard or read about but never met. The most telling evidence in this regard is that when we finally meet someone we have been led to believe is, say, unusually charismatic and compelling, or uncommonly wicked and detestable, we are often “disappointed.” The person often seems less worthy of positive or negative regard than we had been led to expect. Sharpening and leveling can help explain this phenomenon.

Furthermore, an analysis of the accounts provided by the first generation subjects indicated that they did indeed underemphasize the situational determinants of the target person’s actions.

Aspects of the target person’s dispositions were sharpened, whereas features of the surrounding context were leveled.

a number of the textbook authors never read the original research reports. This is a common problem of the academic world: What is implicitly presented as secondhand is often more remote.

Academics often don't read the original paper dealing with the research in question - leading to sharpening and levelling.

An audience may consider a message to be uninformative if it contains too many qualifications, and, as a result, a speaker may be inclined to omit them. This is often seen when scientific findings are reported in the news media: Promising developments are sometimes reported with important qualifications buried in remote parts of the text or omitted altogether.

Media reporting of scientific findings omit qualifications because these are not seen as so important.

Other times, however, people knowingly provide misinformation in the service of what they believe to be “the greater truth.”

Our appetite for entertainment is enormous, and it has a tremendous impact on the tales we tell and the stories we want to hear.

The desire to entertain often creates a conflict for the speaker between satisfying the goal of accuracy and the goal of entertainment.

The decision to stretch the truth is often made unilaterally, and the inaccuracies and distortions are foisted on what is frequently an unsuspecting audience.

The most common type of story that is accepted and spread because of its plausibility is one that is also entertaining and not particularly serious.

The data of our own experience are often biased and incomplete, and we cannot always be counted on to evaluate them fairly. Consequently, those who study human judgment and decision-making urge us to give less weight to our own impressions and to assign more weight to the “base rate,” or general background statistics. For instance, in contemplating the odds that our own marriages might end in divorce, we should attach less significance to our present passion and current conviction that we have found the right person, and we should pay more attention to the overall divorce rate of approximately 50%. To be sure, we should not discount our current feelings and self-knowledge altogether; we just need to temper them a bit more with our knowledge of what happens to people in general. This is the consensus opinion of all scholars in the field. Because personal experience is not an infallible guide to the truth, we must augment it (augment it more than we apparently do) with relevant background statistics.

More generally, the information presented in this chapter indicates that it can be extremely difficult to get a truly accurate estimate of the relevant base-rate. We generally do not collect the base-rate data ourselves; it must be obtained from secondhand sources.

We must learn to distrust personal experience a bit when it conflicts with the base-rate; but when the base-rate comes from an uncertain secondhand source, we must also distrust it when it conflicts with personal experience. Conflict between these two important but imperfect sources of information should temper our judgments and beliefs.

Attaching special significance to the words of the true experts, however, is not as easy as it might seem because reporters will often distort what an expert really said. A common way of doing so is to place an innocuous quote by a credible person next to an outlandish claim, and thus make it appear that the former endorsed the latter. For example: “One source claims that one in three teenagers could be addicted to cocaine within the next five years. Says Elliot Ness, a member of the President’s commission on drug abuse, “There are no easy solutions in the drug war.’” Mr. Ness might indeed believe that there are no easy solutions to the drug war, but nonetheless want to have nothing to do with the “one in three” estimate. The proximity of the two statements in the same paragraph, however, makes it easy to think he is responsible for both. Look carefully at what is actually quoted and what is only implied.

Be on the Lookout for Sharpening and Leveling. Scientists rarely make exact predictions. For instance, rather than stating that “54% of the electorate favors a tax on imported oil,” they will say that “54% plus or minus four percent favor an oil import tax.” Scientific predictions are almost always given as a range or “confidence interval.”

Be Wary of Testimonials. Often the media tries to impress us with the seriousness of some problem by presenting a vivid testimonial of an individual who suffers from it. These accounts are enormously successful in getting us to imagine what it would be like to be in similar circumstances, and they make us much more sympathetic toward those who suffer some bad fate. And well they should! By themselves, however, there is no reason they should have much influence on our sense of the prevalence of some malady.

The false consensus effect refers to the tendency for people’s own beliefs, values, and habits to bias their estimates of how widely such views and habits are shared by others.

It is important to emphasize at the outset the relative nature of the false consensus effect. People do not always think that their own beliefs are shared by a majority of other people. Rather, the false consensus effect refers to a tendency for people’s estimates of the commonness of a given belief to be positively correlated with their own beliefs.

Also consistent with this explanation are results indicating that people are particularly likely to exaggerate the extent to which attractive, respected, and well-liked people have beliefs similar to their own.8

Note that this interpretation of the false consensus effect rests on two assumptions: a) different people construe the same choices quite differently, and b) people generally fail to recognize this fact and thus fail to make adequate allowance for it when making consensus estimates.

Next to the indeterminacy principle, I have learned in recent years to loathe most the term “holistic,” a meaningless signifier empowering the muddle of all the useful distinctions human thought has labored at for two thousand years. Roger Lambert, in John Updike’s Roger’s Version

In fact, many advocates of alternative health practices completely reject controlled experimentation as a valid means for arriving at the truth.

Conventional practitioners might initially defend their pet treatments by explaining away their failures, but most at least acknowledge the supremacy of scientific investigation. Under such scrutiny, a treatment’s weaknesses will eventually come to light, as it has with bloodletting, laetrile, and the porta-caval shunt (see Chapter 10).

Explaining away obvious failures is really just part of a broader tendency to evaluate treatment outcomes in a biased manner. Evidence indicating that a favored practice might be effective is considered decisive; information to the contrary is critically scrutinized and explained away.

A brief anecdote illustrates, perhaps, that even two-time Nobel Prize winners can be misled by the juggling of ambiguous criteria. Linus Pauling, a long-time proponent of vitamin C as an antidote to the common cold and other physical ailments, was once asked whether it was true that he and his wife (who, of course, make sure they consume the requisite amount of the vitamin) no longer suffer from colds. “It is true,” he said, “We don’t get colds at all.” Then he added, “Just sniffles.”16

One such general theory (so general, in fact, it is perhaps best considered a metatheory) is the representativeness heuristic discussed in Chapter 2. According to this overarching belief, effects should resemble their causes, instances should resemble the categories of which they are members, and, more generally, like belongs with like. In the realm of health, this results in the belief that the symptoms of a disease ought to resemble or in some way suggest its cause. Similarly, the symptoms of a disease ought to resemble or in some way suggest its cure.

Representativeness as a meta-theory.

Simple theorizing has also contributed to the widespread belief that we should periodically “cleanse” the insides of our bodies. Just as we periodically clean our car engines or our videocassette recorders to make them function more effectively, so it is believed that our alimentary canals could benefit from an occasional housecleaning as well. Some people do so by fasting, others by administering enemas, drinking large quantities of water, or eating yogurt. Perhaps the most extreme manifestation of this belief was the surgical procedure known as “Lane’s kink.” The British physician Arbuthnot Lane was concerned, along with many of his colleagues, about the deleterious consequences of “auto-intoxication,” or the build-up of waste products in the body. Dr. Lane believed that he had spotted a location in the colon in which the flow of waste slowed down, and so he developed a surgical procedure to cut it out and speed up elimination, a procedure he performed on hundreds unfortunate enough to seek out his services.

The “Up” Side of Holistic Medicine. If we ignore these latter, demonstrably bogus interventions, there is surely some merit to both the underlying philosophy and many of the specific practices of holistic medicine. The emphasis on taking responsibility for the direction of one’s own treatment, for example, is certainly wise. No matter how concerned and compassionate a doctor might be (and not all of them excel at this part of their job), they cannot be as concerned as the patients themselves.

Most of the gain is attributable to various preventive measures such as improved sewage disposal, water purification, the pasteurization of milk, and improved diets. In fact, our greater longevity is mainly due to our increased chances of surviving childhood, chances increased by these very preventive measures and by the introduction of vaccines for the infectious diseases of youth.

of an influential textbook on holistic nursing who states that “Illness occurs when people don’t grow and develop their potentials.”25 Similarly, New Age faith healer Elizabeth Stratton argues that “disease is merely a symptom of a deep psychological problem that the person probably isn’t even aware of…. What I look for is why they created the illness and why they’re hanging on to it.”26 Finally, Eileen Gardner, who served for a brief period in the Reagan administration as an aide to Education Secretary William Bennett, once wrote that handicapped individuals “… falsely assume that the lottery of life has penalized them at random. This is not so. Nothing comes to an individual that he has not, at some point in his development, summoned.” She also claimed that, “As unfair as it may seem, a person’s external circumstances do fit his level of inner spiritual development.”27 This is not exactly the philosophy that one would want in the upper reaches of the Department of Education, the department that is responsible for overseeing educational opportunities for the handicapped.

There really are two classes of self-handicapping strategies, real and feigned. “Real” self-handicapping involves placing visible obstacles to success in one’s own path. The obstacles make one less likely to succeed, but they provide a ready excuse for failure. The student who neglects to study before an exam or the aspiring actor who drinks before an audition are good examples. Sometimes failure is all but guaranteed, but at least one will not be thought to be lacking in the relevant ability (or so it is hoped).

“Feigned” self-handicapping, on the other hand, is in certain respects a less risky strategy, one in which the person merely claims that there were difficult obstacles in the path to success.

Real and feigned self-handicapping strategies.

Self-handicapping is just one example of a class of social strategies people employ to boost their status or achieve some goal, but that in fact often backfire. Name-dropping, boasting, and “coming on strong” are other examples of social strategies that are generally ineffective but are frequently employed.

People think that sufficient quantity can compensate for lack of quality.

A scientific panel commissioned by the National Research Council to study this area concluded that “… despite a 130-year record of scientific research on such matters, our committee could find no scientific justification for the existence of phenomena such as extrasensory perception, mental telepathy, or ‘mind over matter’ exercises….