Naked Statistics: Stripping the Dread from the Data

by Charles Wheelan

Illinois, the probabilities associated with the various possible payoffs for the game are printed on the back of each ticket.

The law of large numbers explains why casinos always make money in the long run. The probabilities associated with all casino games favor the house (assuming that the casino can successfully prevent blackjack players from counting cards).

“probability density functions” for a Schlitz type of test with 10, 100, and 1,000 trials.

converges around 50 percent of tasters’ choosing Schlitz as the number of tasters gets larger. At the same time, the probability of getting an outcome that deviates sharply from 50 percent falls sharply as the number of trials gets large.

number of tasters gets larger: 10 blind taste testers: .83 100 blind taste testers: .98 1,000 blind taste testers: .9999999999 1,000,000 blind taste testers:

Okay, maybe that’s not so obvious. Let me back up. The entire insurance industry is built on probability. (A warranty is just a form of insurance.)

Buffett can save money by not purchasing car insurance, homeowner’s insurance, or even health insurance because he can afford whatever bad things might happen to him.

Which finally brings us back to your $99 printer! We’ll assume that you’ve just picked

The book Data Mining and Predictive Analysis: Intelligence Gathering and Crime Analysis,

Probability gives us tools for dealing with life’s uncertainties. You shouldn’t play the lottery. You should invest in the stock market

I have in mind “Six Sigma Man.” The lowercase Greek letter sigma, σ, represents the standard deviation. Six Sigma Man is six standard deviations above the norm in terms of statistical ability, strength, and intelligence.

Yes. The contestant has a 1/3 chance of winning if he sticks with his initial choice and a 2/3 chance of winning if he switches. If you don’t believe me, read on.

My second explanation gets at the intuition. Let’s suppose the rules were modified slightly.

In both cases, switching gives you the benefit of two doors instead of one, and you can therefore double your chances of winning, from 1/3 to 2/3.

My third explanation is a more extreme version of the same basic intuition. Assume that Monty Hall offers you a choice from among 100 doors rather than just three. After you

Of course you should. There is a 99 percent chance that the car was behind one of the doors that you did not originally choose.

applicable lesson is that your gut instinct on probability can sometimes steer you astray.

CHAPTER 6 Problems with Probability How overconfident math geeks nearly destroyed the global financial system

The model assumed that there is a range of possible outcomes for every one of the firm’s investments.

Not necessarily. VaR has been called “potentially catastrophic,” “a fraud,” and many other things not fit for a family book about statistics like this one.

Unfortunately, there were two huge problems with the risk profiles encapsulated by the VaR models. First, the underlying probabilities on which the models were built were based on past market movements; however, in financial markets (unlike beer tasting), the future does not necessarily look like the past. There was no intellectual justification for assuming that the market movements from 1980 to 2005 were the best predictor of market movements after 2005. In some ways, this failure of imagination resembles the military’s periodic mistaken assumption that the next war will look like the last one. In the 1990s and early 2000s, commercial banks were using lending models for home mortgages that assigned zero probability to large declines in housing prices.2

Second, even if the underlying data could accurately predict future risk, the 99 percent assurance offered by the VaR model was dangerously useless, because it’s the 1 percent that is going to really mess you up.

“This is like an air bag that works all the time, except when you have a car accident.” If a firm has a Value at Risk of $500 million, that can be interpreted to mean that the firm has a 99 percent chance of losing no more than $500 million over the time period specified.

“tail risk,” the small risk (named for the tail of the distribution) of some catastrophic outcome. (If

The fact that you’ve never contemplated that your town might be flattened by a massive asteroid was exactly the problem with VaR.

Nicholas Taleb, author of The Black Swan: The Impact of the Highly Improbable and a scathing critic of VaR: “The greatest risks are never the ones you can see and measure, but the ones you can’t see and therefore can never measure. The ones that seem so far outside the boundary of normal probability that you can’t imagine they could happen in your lifetime—even though, of course, they do happen, more often than you care to realize.”

VaR debacle is the opposite of the Schlitz example in Chapter 5. Schlitz was operating with a kno...

quants made three fundamental errors. First, they confused precision with accuracy. The VaR models were just like

exact and wrong.

Assuming events are independent when they are not.

Not understanding when events ARE independent. A different kind of mistake occurs when events that are independent are not treated as such.

Amos Tversky,

chance.” We see patterns where none may really exist. Like cancer clusters.

Clusters happen.

(Despite my general aversion to lotteries, I do admire the Illinois slogan: “Someone’s gonna Lotto, might as well be you.”) Here is an exercise

When we see an anomalous event like that out of context, however, we assume that something besides randomness must be responsible.

The prosecutor’s fallacy. Suppose you hear testimony in court

Reversion to the mean (or regression to the mean).

worse (though I would not necessarily rule that out);

At first glance, reversion to the mean may appear to be at odds with the “gambler’s fallacy.”

And the more flips, the more closely the outcome will resemble the 50-50 mean outcome that the law of large numbers predicts.

“Our results suggest that media-induced superstar culture leads to behavioral distortions beyond mere mean reversion.” In other words, when a CEO appears on the cover of Businessweek, sell the stock.

Statistical discrimination.

To insurers, however, gender-based premiums aren’t discrimination; they’re just statistics.

Yep, I used the profiling word, because that’s the less glamorous description of the predictive analytics that I described so glowingly in the last chapter, or at least one potential aspect of it.

But what we can or should do with that kind of information (assuming it has some predictive value) is a philosophical and legal question, not a statistical one. We’re getting more and more information every day about more and more things.

We like to think of numbers as “cold, hard facts.” If we do the calculations right, then we must have the right answer. The more interesting and dangerous

The Importance of Data “Garbage in, garbage out”

Since I am not an expert in this field, I had two slightly different reactions upon reading about spurned fruit flies. First, it made me nostalgic for college. Second, my inner researcher got to wondering how fruit flies get drunk.

Data are to statistics what a good offensive line is to a star quarterback.