From the Bookshelf of Science and Inquiry…

The Signal and the Noise: Why So Many Predictions Fail—But Some Don't

Nate Silver

Start date: March 1, 2013

Finish date: March 31, 2013

Discussion: Book Club 2013

Find A Copy At

Libraries

Group Discussions About This Book

Showing 2 of 26 topics — 537 comments total

Book Club 2013

December 2013: "The Strangest Man"
By David · 22 posts · 150 views

last updated Jan 03, 2016 10:51AM

June 2013 - Cosmos
By Betsy , co-mod · 45 posts · 171 views

last updated Mar 11, 2014 04:07PM

showing 2 of 2 topics view all »

What Members Thought

Apr 18, 2013 David Rubenstein rated it it was amazing

Shelves: environment, geology, mathematics, science, economics, meteorology

This is a fantastic book about predictions. I enjoyed every page. The book is filled to the brim with diagrams and charts that help get the points across. The book is divided into two parts. The first part is an examination of all the ways that predictions go wrong. The second part is about how applying Bayes Theorem can make predictions go right.

The book focuses on predictions in a wide variety of topics; economics, the stock market, politics, baseball, basketball, weather, climate, earthquakes This is a fantastic book about predictions. I enjoyed every page. The book is filled to the brim with diagrams and charts that help get the points across. The book is divided into two parts. The first part is an examination of all the ways that predictions go wrong. The second part is about how applying Bayes Theorem can make predictions go right.

The book focuses on predictions in a wide variety of topics; economics, the stock market, politics, baseball, basketball, weather, climate, earthquakes, chess, epidemics, poker, and terrorism! Each topic is covered lucidly, in sufficient detail, so that the reader gets a good grasp of the problems and issues for predictions.

There are so many fascinating insights, I can only try to convey a few. At the present time, it is impossible to predict earthquakes, that is, to state ahead of time when and where a certain magnitude earthquake will occur. But it is possible to forecast earthquakes in a probabilistic sense, using a power law. Likewise, it may be possible to forecast terrorism, because that too, follows a power law! (Well, it follows a power law in NATO countries, probably because of the efforts to combat terrorists. But in Israel, the tail of the curve falls below the power law, likely because of the stronger anti-terror emphasis there.)

The accuracy of weather predictions increases slowly but steadily, year by year. Ensembles of computer model runs are part of the story, but human judgment add value, and increases the accuracy. Weather forecasts issued by the National Weather Service are unbiased in a probabilistic sense. But weather forecasts by the TV weatherman are very strongly biased--the weatherman over-predicts precipitation by a significant amount.

Nate Silver shows that the people who are most confident are the ones that make the worst predictions. The best predictions are those that are couched in quantitative uncertainties. Silver shows how Bayes Theorem can be applied to improve predictions; it is all about probabilities. And I just love this footnote,

A conspiracy theory might be thought of as the laziest form of signal analysis. As the Harvard professor H.L. "Skip" Gates says, "Conspiracy theories are an irresistible labor-saving device in the face of complexity."

...more

35 likes · Like · see review

Dec 25, 2012 Casey rated it really liked it · review of another edition

Shelves: knowledge-is-good, relevant-for-teachers

3.5 stars, but I'm rounding up because I like stats. The first half of this book is fantastic: it outlines the issues that cause people to make terrible predictions. Across many fields, people are not so good at prediction, for a number of reasons. Silver fights the idea that having enough data means that predictions will be great. Data is noisy, and just adding more noisy data isn't going to allow computers to magically find signal. There's a reason why statisticians say "Garbage in, garbage ou 3.5 stars, but I'm rounding up because I like stats. The first half of this book is fantastic: it outlines the issues that cause people to make terrible predictions. Across many fields, people are not so good at prediction, for a number of reasons. Silver fights the idea that having enough data means that predictions will be great. Data is noisy, and just adding more noisy data isn't going to allow computers to magically find signal. There's a reason why statisticians say "Garbage in, garbage out."

The second half of the book, which focuses on the so-called solution, is where everything falls apart. According to Silver, the answer is Bayes theorem. I don't take much issue with this: Bayes theorem is elegant and useful, particularly when it comes to making inferences about the world. There's a reason why you can't go to a conference on cognition without hearing a talk on Bayesian learning: it's a good idea, and it seems to work.

The problem is that Silver seems to confuse Bayesian statistics with "thinking probabilistically" when the two don't really mean the same thing. I think probabilistically when I get on a flight (or walk past the lottery ticket counter without buying one). The probability of a safe flight is close to 100%, whereas the probability of winning the lottery is close to 0%. This is not Bayesian, it's just a simple computation of some event of interest divided by total events (e.g., safe flight / all flights (safe and..., well, not so good)).

Bayes theorem is different. Most simply, it's stated as follows: P(A|B) = [P(B|A) * P(A)] / P(B). As you may or may not be able to discern from the equation, it allows you to compute the probability of some event A occurring given that some event B has occurred, using the probability that B occurs given A, the probability of A on its own, and the probability of B. In simple terms, imagine that I'm waiting on the elevated platform for my train. When I get to the platform, I don't have much reason to believe that my train will go express and skip my stop. However, I wait for awhile, longer than I should have to, and the train doesn't come. I can compute the probability that the long wait signals that I'll get skipped using Bayes theorem, and it's possibile that my brain has been doing something Bayesian throughout my many morning commutes. Give it a little input, and Bayes will let me know if I'm better off taking a cab.

Throughout The Signal and the Noise, Silver will profile someone who's making a lot of money betting on sports, or making a lot of money playing poker. Then, he'll say "And this is Bayesian! This proves that we should all be using Bayesian stats all the time!" He doesn't really explain how any of these people are applying Bayesian stats, or profile anyone making good predictions using a non-Bayesian approach. I like Bayes, but Silver is going to have to do more than say "Look! Bayes!" to convince me that it's the panacea for prediction.

Silver also points out that conventional methods for hypothesis testing produces far more false-positives (usually called Type I errors) than they should. This is true, and most scientists are aware that they need to make some changes in the way they analyze data. Silver, of course, thinks that we should all be using Bayes, although he doesn't speak about any of the other methods that can reduce Type I errors (including simple methods, like reporting the effect size). At one point, Silver equates science with Bayesian thinking, while completely missing the utility of a well-designed experiment.

I do recommend Silver's book, particularly for non-scientists who are interested in statistics. However, it's best taken a series of interesting stories about prediction, and not as a fully developed theory for how most predictions should be developed. ...more

3 likes · Like · see review

Oct 15, 2012 bup rated it it was amazing

Shelves: 2013, science, favorites, math

In an imaginary world where people self-motivate to become better people, and to spend each day moving toward substance and moving away from fluff and appearance, this book would be read by every person in the world.

That's frustrating, but at least the people who do read this book can make the world a little better. Seriously. Reading this book will make the world a better place.

Understanding that employing probabilities is wise, and that a demonstration of confidence in knowledge is usually a s ...more

2 likes · Like · see review

Aug 26, 2013 Mitchell Friedman rated it liked it

Shelves: economics, nonfiction, mathematics, library, liked-reviews, published-2012

This book is currently being discussed at work chapter by chapter. I'm not really sure why this one rather than some other book was selected. It's kind of a survey of multiple topics. And certainly a simple enough book that us less math-y types could follow it. But it just didn't grab me. And didn't have enough new-ish content. It did do a good job on getting across some ideas that I've had trouble with in the past including Bayes Law and Power Law Distribution and Double Log plots. But it also ...more

1 like · Like · see review

Jul 31, 2013 Justarius rated it it was ok

The book has an interesting premise (how to use statistics to make better predictions), but it did not deliver a satisfactory experience. There are two reasons for this. First, the book is mainly a collection of case studies ranging from science to baseball to poker. Many of these stories are interesting, but they do not always provide helpful insights related to the premise. Second, the proposed solution (use stats but exercise good judgement on how to use those stats) is essentially common sen ...more

Like · see review

Sep 17, 2012 Charles rated it really liked it

A very interesting book. I learned about Bayes’ rule. I never really understood what it was before. Silver did a good job promoting Bayes' idea. I personally like Bayes’ philosophy. I have a suspicion why academia doesn't really teach Bayesian probability. I suspect it's because probabilistic approach weakens the science. Science loves theorems. Something that is provable. They don't want to hear this theory has 80% chance that it's correct. Truths cannot be probabilistic in their mind. It’s abs ...more

Like · see review

Sep 09, 2014 Rod rated it liked it

Shelves: science, economics

The theme of this book is that we're not as good at predicting the future as we think we are. No kidding. I predicted I would really like this book.

Seriously, I do recommend the book. Though it does have some flaws and shortcomings, it is a good overview of the impact our poor understanding of probability has on our lives. The sections on sports, the stock market, and poker are insightful. Sections on global warming and terrorism, not so much. For example, the author interviews the execrable Mic ...more

Like · see review