The Skeptics' Guide to the Universe: How to Know What's Really Real in a World Increasingly Full of Fake

by Steven Novella

There is also a statistical phenomenon called regression to the mean. In any varying system, which could be athletic performance or the waxing and waning symptoms of a chronic illness, any extreme variance is statistically likely to be followed by a more typical variance. This means that when you have bad symptoms, it’s likely that your symptoms will eventually become milder, or regress to the mean. This also means that any treatment you take when your symptoms are severe is likely to be followed by a lessening of those symptoms, creating the illusion that the treatment worked.

There is no method to directly measure pain as a phenomenon, and studies of pain are dependent upon the subjective reports of those being treated. There is therefore a large potential for perception and reporting bias in pain trials. But there are biological mechanisms by which mental processes can affect pain. For example, increased physical activity can release endorphins that naturally inhibit pain. The perception of pain can also be decreased by simple distraction. Even cursing reduces reported pain intensity. For these reasons the placebo effect for pain is typically high, around 30 percent.

The more concrete and physiological the outcome, the smaller the placebo effect.

Any real benefits that contribute to the placebo effect can be gained by more straightforward methods—like healthy habits, compliance with treatment, and good health care. The placebo effect isn’t evidence for any mysterious mind-over-matter effect, but since the mind is matter (the brain) and is connected to the rest of the body, there are some known physiological effects that do play a role (although often greatly exaggerated).

Keep in mind, you can get the same placebo effects from science-based treatments. You don’t need to believe in magic.

Once scientific studies show their treatments are no better than placebo (meaning they don’t work), then the proponents claim that this is okay, because placebos work too.

In a sense, much of so-called “alternative medicine” practice is placebo medicine—using elaborate rituals with fanciful explanations to produce nothing but imaginary placebo effects.

28.

Skeptics’ Guide Entry: Anecdote

Section: Science and Pse...

See also: Anecdotal...

An anecdote is a story or experience, often offered as evidence, that was not controlled and is therefore subject to a host of biases and confounding factors. It is considered the weakest form of evidence, ...

Scientists and skeptics have come to use the word “anecdotal” as a derogatory dismissal of certain kinds of weak evidence—and with good reason.

One of the hard-won lessons of the process of scientific discovery is that anecdotal evidence is very unreliable. Anecdotes can be worse than worthless because they can be misleading.

After being hit on the head so many times by anecdotal evidence throughout history, modern scientists are now appropriately wary.

What we mean, exactly, by anecdotal evidence is the report of an experience by one or more persons that is not objectively documented, or an experience or outcome that occurred outside of a controlled environment. Such evidence is unreliable because it depends upon the accurate perception of the witness(es), often in a situation where the event was unexpected or unusual; it is dependent upon subjective memory, which has overwhelmingly proven to be extremely error prone and subject to a host of flaws; and it cannot account for the random vagaries of a chaotic world.

At best, anecdotes can be used as an indication of a possible (not even probable) phenomenon that is deserving of further research or exploration. It can be used to generate a hypothesis, but it shouldn’t be used as a basis to confirm one.

We learn to trust our anecdotal experience for a few reasons. For everyday simple things, it’s a simple and efficient approach.

We are also predisposed to believe stories, especially when they have an emotional theme.

Much like our ancient ancestors, we might be tempted to accept an anecdotal report from an alleged “credible witness.” But the credible witness is now just as much a myth as Bigfoot. We are all humans. We all have flawed and biased brains. Being an airline pilot, for example, doesn’t change the way your brain processes visual information. Some witnesses are certainly better than others, but any witness can be fooled.

It’s common for eyewitnesses, especially if they were together at the time, to talk during and after the experience about what they were seeing. The perceptions of one person can thereby easily contaminate the memory of the other witnesses.

The bottom line is this—we know for certain from countless historical examples that even people who have all the traits of a reliable witness can be profoundly mistaken.

Why Would They Lie?

Clearly, whether or not someone believes what they are reporting says nothing about the accuracy of the information—just that they’re not willfully lying.

The inability to figure out why someone might lie is no reassurance that they are in fact telling the truth (an argument from ignorance, right?).

ICONIC CAUTIONARY TALES FROM HISTORY

The same mistakes, errors in thinking, biases, and cons keep getting recycled. The skeptical lessons of the past are all still relevant today, and studying their history is an easy way to learn from the mistakes of others.

All the pseudosciences we confront today have their roots in the past.

29. Skeptics’ Guide Entry: The “Clever Hans” Effect

Section: Cautionary Tales

See also: Pseud...

The Clever Hans effect refers to an unconscious nonverbal communication of information from a researcher or agent to an animal or other subject. This unconscious communication creates t...

Pfungst concluded that during the extensive training of Hans, the horse learned to pick up on subtle cues from his trainer when he hit the target number of stomps on the ground. So he would just keep stomping until he got the signal. There is no indication that von Osten was aware he was giving such cues, and in fact Hans was able to pick up on cues from people other than his trainer.

A Cautionary Tale

In cases of extreme scientific implausibility, it’s perhaps best to withhold judgment until truly expert investigation can be done and results can be independently replicated. Such a position isn’t closed-minded, it is simply realistic. If something seems too fantastical to be true, history tells us it is extremely likely that it’s not true, that it’s just an illusion, an error, or an artifact of flawed methodology. At the very least, all such possibilities should be ruled out to a degree that is in proportion with the implausibility of the claim.

The lesson of Clever Hans is critical to keep in mind the next time a gushing reporter or enthusiastic researcher promotes the apparently impossible. Such stories often come with the challenge to “Explain that, skeptic!” Give it time.

30. Skeptics’ Guide Entry: The Hawthorne Effect

Section: Cautionary Tales

See also: The Observ...

The Hawthorne or observer effect refers to the fact that simply observing something may alter its behavior, thereby creating an artifact ...

We tend to understand the world through distilled narratives, simple stories that approximate reality (whether we know it or not or whether we intend to or not).

The Hawthorne Effect

The distilled narrative of the Hawthorne effect is this: The act of observing people’s behavior changes that behavior.

The problem is that most one-paragraph summaries in the public consciousness are not essentially correct. Sometimes more detail and nuance are needed to get to an acceptable level of correctness. Knowing where that level resides is part of the skill of science communication.

Results

Conclusion

How to Interpret All This?

Teachers have a great influence on the performance of their students, although it’s still not clear exactly what the mechanisms are. That influence is biased by expectations, which is exactly why so much attention is paid to gender and race biases in teachers. It’s also why we have to be very careful with labels and with using possibly biased standardized tests to establish expectations.

The very notion that someone upstairs cares about details like the light level in your working space may have a profound effect on the work culture.

Workers also influence each other. They establish norms of behavior and expectations for standards and productivity.