Naked Statistics: Stripping the Dread from the Data

by Charles Wheelan

WHAT IS THE FUTURE OF FOOTBALL?

2009, Malcolm Gladwell

Here is what we know. There is mounting evidence that concussions and other brain injuries associated with playing football can cause serious and permanent neurological damage. (Similar phenomena have been observed in boxers and hockey players.)

Perhaps the most poignant was Dave Duerson, a former safety and Super Bowl winner for the Chicago Bears, who committed suicide by shooting himself in the chest; he left explicit instructions for his family to have his brain studied after his death.

impacts of brain trauma is Ann McKee, who runs the neuropathology laboratory at the Veterans Hospital in Bedford, Massachusetts. (Coincidentally, McKee also does the neuropathology work for the Framingham Heart Study.)

According to his data, players routinely receive blows to the head with a force equivalent to hitting the windshield in a car crash at twenty-five miles per hour.

WHAT (IF ANYTHING) IS CAUSING THE DRAMATIC RISE IN THE INCIDENCE OF AUTISM?

The first intriguing statistical question is whether we are experiencing an epidemic of autism, an “epidemic of diagnosis,” or some combination of the two?6

In any case, the shockingly high incidence of ASDs represents a serious challenge for families, for schools, and for the rest of society. The average lifetime cost of managing an autism spectrum disorder for a single individual is $3.5 million.

One of the most important contributions of statistical analysis so far has been to debunk false causes, many of which have arisen because of a confusion between correlation and causation.

appears suddenly between a child’s first and second birthdays.

Scientists have soundly refuted the false association between thimerosal and ASDs. Autism rates did not decline when thimerosal was removed from the MMR vaccine, nor are autism rates lower in countries that never used this vaccine.

HOW CAN WE IDENTIFY AND REWARD GOOD TEACHERS AND SCHOOLS?

This appears to be a case where everybody is right—up to a point. Doug Staiger, an economist at Dartmouth College who works extensively with value-added data for teachers, warns that these data are inherently “noisy.”

indicators is only about .35. (Interestingly, the correlation in year-to-year performance for Major League baseball players is also around .35, as measured by batting average for hitters and earned run average for pitchers.)14

The teacher effectiveness data are useful, says Staiger, but they are just one tool in the process for evaluating teacher performance. The data get “less noisy” when authorities have more years of data for a particular teacher with different classrooms of students (just as we can tell more about an athlete when we have data for more games and more seasons).

We like rankings—just think U.S. News & World Report college rankings—even when the data do not support such precision. Staiger offers a final warning

Which professors are most effective?

The answer: The professors with less experience and fewer degrees from fancy universities.

Meanwhile, the old, crusty professors (whom we nearly fired just one paragraph ago) focus less on the exam and more on the important concepts, which are what matter most in follow-on courses and in life after the Air Force Academy.