When Can You Trust the Experts?: How to Tell Good Science from Bad in Education

by Daniel T. Willingham

The Golden Ratio does exert a powerful and powerfully subtle influence on persuasion. Or it would, if not for a small problem: the Golden Ratio theory is bunk.

I apologize for beginning this book with a sucker punch. (Maybe some part of me wanted company. I fell for the Golden Ratio hook, line, and sinker when I first heard it.a) The Golden Ratio is not interesting because it’s true. It’s interesting because the idea survives and continues to attract believers even though it is known to be wrong. In that way, it’s an object lesson for this book. Knowing what to believe is a problem.

Consider learning styles theories.

If the learning styles theory is true, people who experienced the story in their preferred way ought to remember it better. Experiments like this have been conducted, and there is no support for the learning styles idea.7 Not for visual, auditory, or kinesthetic learners, nor for linear or holistic learners, nor for any of the other learners described by learning styles theories.

Before about 1920, the way to teach children to read seemed obvious.

The whole-word method became dominant in American education during the 1930s and 1940s.

Copying an expert reader is not necessarily a good strategy for beginning readers.

!

Was phonics-based or whole-word instruction superior? Jeanne Chall, a professor at the Harvard Graduate School of Education, was selected to conduct the review. In her 1967 book, she said that the relevant research showed that the phonics method was superior.13

heuristic. A heuristic is a shortcut. It’s not the best way to do something, but it yields a solution that’s usually pretty good, and it has the great benefit of practicality: it’s easy to calculate.

strip it and flip it; trace it; and analyze it;

Similarity aids persuasion even when based on something as trivial as having the same nervous tic or taking an ice cream sample of similar size.5 We unconsciously imitate each other to smooth social interactions.

What are the cues that tell our inattentive mind “This message is probably true”?

Familiar Ideas Are More Believable

We Believe Things That Others Believe

We Believe Attractive People

We Believe People Who Are Like Us

Peripheral features include things like the familiarity of the message, how it makes us feel, the attractiveness of the source of the message, whether we identify with him, and his apparent expertise.

We seek to persuade ourselves that our beliefs have always been correct and that the new information before us merely confirms what we already knew. This tendency is called the confirmation bias, and it affects all stages of thinking: what information we seek, how we interpret information when we find it, and how we remember it later.

The confirmation bias will make me more likely to pose questions that assume the applicant is an introvert, and the person will come off looking like one.24 Worse yet, suppose I’m a physician, and a few symptoms lead me to suspect that a patient has a particular disease. Might not the confirmation bias lead me to order tests that might confirm my diagnosis, instead of other tests? The answer is yes,25 although more experienced doctors may be better at resisting this tendency.26

We see what we think we’ll see.

The bigot who thinks, for example, that African Americans are lazy will tend to notice and remember any instance of laziness he observes in African Americans. Hence, the bigot will note (and remember) an encounter with a lackadaisical store clerk who is black, but the same interaction with a white clerk will go unnoticed, or the bigot will assume that the clerk has a valid excuse for being a little slow.28

It seems that when we encounter a conclusion we disagree with, our minds spring into action, looking for flaws in the argument. But if we agree with someone, we’re more likely to say to ourselves, “Yes, yes, I already know this. I’m so glad you agree with me.”f 32

It would be disruptive indeed if you changed your beliefs every time you encountered a new bit of evidence. I say “disruptive” because very few of our beliefs are wholly isolated. For example, my belief that global warming is a serious problem is connected to my belief that I was smart and virtuous to buy a hybrid car. It’s also connected to my dislike for my coworker who is full of loud scorn for global warming. So if I change my belief about global warming, that affects my belief about my car (I was a sucker to pay extra for a “green” car) and about my coworker (that loudmouth was right all along).35

Our Beliefs Help Maintain Our Self-Identity

Our Beliefs Help Protect Our Values

Our Beliefs Help Maintain Social Ties

Our Beliefs Help Us Manage Our Emotions

This chapter has been a parade of disappointing facts, easily summarized: when we don’t weigh evidence carefully, we are prone to believing or disbelieving things for trivial reasons; and even when we do weigh evidence carefully, we are still subject to those trivial influences.

Chapter 2 Science and Belief A Nervous Romance

But the best demonstration by far is experience. —Francis Bacon1 Sweet is the lore which Nature brings Our meddling intellect Mis-shapes the beauteous forms of things We murder to dissect —William Wordsworth

In Chapter One, we saw that we believe things (or not) partly due to the presence of peripheral cues—“peripheral” meaning that they are aspects of the situation that are irrelevant to whether or not the message is really true (for example, the attractiveness of the speaker or the length of the message). Although we are seldom aware that they influence us, these cues have an impact even when we consciously try to evaluate the logic and factual basis of a persuasive message. We also saw that we have a broad bias to believe new information if it is consistent with what we already believe, and we criticize new information that is inconsistent with our beliefs. To protect ourselves from believing false things, we need to (1) be aware of the peripheral cues that persuade, so that we can discount them, and (2) be aware of the beliefs we now hold, because they will bias how we evaluate new information.

To get started in your evaluation, you need to be very clear on three points: (1) precisely what Change is being suggested; (2) precisely what outcome is promised as a consequence of that Change; and (3) the probability that the promised outcome will actually happen if you undertake the Change.

The Change recommended in Exhibit 5.1 is much harder to strip using the “If I do X, then there’s a Y percent chance that Z will happen” formula, and that’s informative. X is hard to pin down, and Y is unspecified. To be fair, this advice was drawn from a blog entry, so perhaps I shouldn’t take it all that seriously. At the same time, though, you see a lot of advice to teachers that is similarly frothy. Such advice is difficult to disagree with, but it doesn’t help because it is so indefinite.

sarcasm tone in audio book

Politicians don’t persuade with statistics. They persuade with emotions.

2016!

When Persuaders target teachers, they more often use emotional appeals centering on hope, not fear. Most teachers you meet are optimists. They believe that all children can learn and that all children have something to offer the classroom. Teachers are also optimistic about the possibility that they can help children fulfill their potential. An unpublished survey I conducted of several hundred teachers showed that the most frequent response to the question “Why did you become a teacher?” was “I wanted to make a difference in the world.” But teachers are not optimists to the point that they are out of touch with reality. A teacher knows when there is a child with whom she is not connecting. She knows if some aspect of her teaching has become grooved, familiar, and a little stale. When they talk to teachers, Persuaders offer a Change as a way finally to reach that unreachable child or to put the passion back into the teaching. Administrators often try to sell teachers on an idea by dangling hope before them. Administrators know that “buy-in” is vital—if teachers don’t believe that a Change is a good idea, they won’t implement it in their classroom. Thus administrators see a need not merely to persuade teachers but also to inculcate zeal for the Change. Fear does not encourage zeal. It encourages grudging compliance. Hope breeds zeal. That is why professional development sessions sometimes feel like evangelical revival meetings. But hope, like fear, is not a reason to believe...

characteristics of the Persuader are a very weak indicator of scientific credibility. Stripping the claim will help you ignore them.

We are risk averse for gains, and risk seeking for losses.

What I’m emphasizing here is that you should look at these outcomes from all possible angles, because your willingness to try something risky is influenced by whether you think of yourself as trying to get something good or trying to avoid something bad.

Flip it means that you 1. Consider whether the promised outcome can be described another way. If it’s described positively, is there a negative side to it? (For example, a “pass rate” can also be described as a “failure rate.”) The new description may sound fine to you, but it may not. Try it. 2. Consider not just the consequences of undertaking the Change, but the consequences of doing nothing. 3. Combine the two flips. When comparing the consequence of undertaking the Change (versus doing nothing), also be sure to make that comparison with the outcome flipped. That is, compare the Change to doing nothing when you think of the outcome as good, and compare the Change to doing nothing when you think of the outcome as bad.

some claims are, on their face, unworthy of attention: some are boring, some are unacceptably vague, and some are unlikely to affect students.

Little kids learn a lot.

hahaha!