What do you think?
Rate this book
The Improbability Principle: Why coincidences, miracles and rare events happen all the time
Why is it that incredibly unlikely phenomena actually happen quite regularly and why should we, in fact, expect such things to happen? Here, in this highly original book - aimed squarely at anyone with an interest in coincidences, probability or gambling - eminent statistician David Hand answers this question by weaving together various strands of probability into a unified explanation, which he calls the improbability principle.This is a book that will appeal not only to those who love stories about startling coincidences and extraordinarily rare events, but also to those who are interested in how a single bold idea links areas as diverse as gambling, the weather, airline disasters and creative writing as well as the origin of life and even the universe. The Improbability Principle will change your perspective on how the world works – and tell you what the Bible code and Shakespeare have in common, how to win the lottery, why Apple's song shuffling was made less random to seem more random. Oh and why lightning does in fact strike twice...
288 pages, Paperback
First published February 11, 2014
231 people are currently reading
4411 people want to read
About the author
David J. Hand
44 books59 followersDavid J. Hand is Senior Research Investigator and Emeritus Professor of Mathematics at Imperial College, London, and Chief Scientific Advisor to Winton Capital Management. He is a Fellow of the British Academy, and a recipient of the Guy Medal of the Royal Statistical Society. He has served (twice) as President of the Royal Statistical Society, and is on the Board of the UK Statistics Authority. He has published 300 scientific papers and 25 books: his next book, The Improbability Principle, is due out in February 2014. He has broad research interests in areas including classification, data mining, anomaly detection, and the foundations of statistics. His applications interests include psychology, physics, and the retail credit industry - he and his research group won the 2012 Credit Collections and Risk Award for Contributions to the Credit Industry. He was made OBE for services to research and innovation in 2013.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 198 reviews
April 10, 2019
After making my way through The Improbability Principle, I came to a startling revelation: I really have to stop running out and buying books with positive reviews in MacLean’s. I suppose this very concisely shows my opinion on the book, but allow me to explain myself.
The Improbability Principle has a very interesting concept: offering theories as to why the highly improbable seems to happen all the time. I will grant that Hand’s explanations are very insightful, but getting there proved to be highly tedious. Having a rudimentary understanding of statistics and probability, the most painful to read were the basic concepts needed for these explanations. (I was glad to see the most basic of them relegated to the appendices, but I felt more could have been there.)
One of the largest failings of the book, however, is the fact that it strays away from the subject matter quite often, focusing less on the central argument and more on why Hand refuses to believe in miracles, extra-sensory perception, and Creationism. (The rant on Creationism was a particular low point in my mind, given as a very weak argument that should have been expanded upon or omitted entirely.) Things improve when he steers the discussion back to the subject, but, even then, much of it felt padded and verbose.
And there, I believe, is the main source of my dissatisfaction. Many of Hand’s theories come across very concisely, but he goes on and on giving examples and further talking without adding much to the discussion. I will say that many of the improbable statistics and anecdotes presented are enjoyable––to the extent that I frequently had to pause my reading in order to repeat the anecdotes to my wife––but there seem to be far more than necessary to make a point. Parts of The Improbability Principle were highly interesting but, padding removed, the book would be half its current length, in all probability.
The Improbability Principle has a very interesting concept: offering theories as to why the highly improbable seems to happen all the time. I will grant that Hand’s explanations are very insightful, but getting there proved to be highly tedious. Having a rudimentary understanding of statistics and probability, the most painful to read were the basic concepts needed for these explanations. (I was glad to see the most basic of them relegated to the appendices, but I felt more could have been there.)
One of the largest failings of the book, however, is the fact that it strays away from the subject matter quite often, focusing less on the central argument and more on why Hand refuses to believe in miracles, extra-sensory perception, and Creationism. (The rant on Creationism was a particular low point in my mind, given as a very weak argument that should have been expanded upon or omitted entirely.) Things improve when he steers the discussion back to the subject, but, even then, much of it felt padded and verbose.
And there, I believe, is the main source of my dissatisfaction. Many of Hand’s theories come across very concisely, but he goes on and on giving examples and further talking without adding much to the discussion. I will say that many of the improbable statistics and anecdotes presented are enjoyable––to the extent that I frequently had to pause my reading in order to repeat the anecdotes to my wife––but there seem to be far more than necessary to make a point. Parts of The Improbability Principle were highly interesting but, padding removed, the book would be half its current length, in all probability.
October 21, 2016
Excelente livro sobre probabilidade e situações do dia-a-dia. De certa forma, complementar ao O andar do bêbado. Com boas noções e os princípios matemáticos que explicam coincidências extremas, como a lei da improbabilidade, a lei dos números grandes, a lei do próximo o suficiente e outros (tradução porca minha). Com direito a uma viagem pelas chances do surgimento da vida. Leve, fácil de entender e bem explicado, acessível e pensado para ser interessante. Um dos livros mais legais que li esse ano, pela combinação de anedotas e boas explicações.
March 1, 2023
This is the best book on statistics I've ever read, and I've taken several courses on statistics. You can apply the principles to playing the lottery and games of chance (I'm a backgammon player). It will also change your thinking about "coincidences" and improbable events. With enough tries, anything that is possible is likely to happen. (People have won the lottery twice.) And when you investigate deeper, some events might not have been so improbable in the first place. Like the time my wife and I met our next-door neighbors at London's Heathrow Airport. Or the time I met a member of the Michigan NCAA championship basketball team in Los Angeles while I was wearing a T-shirt commemorating the event. Speaking of my next-door neighbor, we have the same birthday, which I always thought was unusual. But the book got me to thinking and I figured out that this probably occurs thousands of times in the Los Angeles area. You'll never look at forecasts the same way either. Forecasts, such as computer forecasts, are based on assumptions. If your assumptions are even slightly off (and they always are) this creates large changes in the results. So the next time someone tells me the ocean level is going to rise by 20 feet I will take it with the proper dose of salt.
October 20, 2015
“No testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that it's falsehood would be more miraculous, than the fact which it endeavors to establish.” David Hume
"Understanding means removing obscurity, obfuscation, ambiguity and confusion” David Hand
The Improbability Principle is the counter punch to Carl Jung’s “synchronicity”. Jung, the creator of “archetypes” used in psychotherapy practice, sought to prove that unusual coincidences (or synchronicity) that he documented with his patients must be due to supernatural causes. Unfortunately, Jung was a psychoanalyst not a statistician. David Hand, on the other hand, uses mathematics and statistics to demonstrate that “coincidences” need no extraordinary cause but are simply a clear effect of the law of very large numbers.
Before jumping into the history of probability, which is where this work shines, Hand demonstrates that there is a wide range of beliefs that run the gauntlet from the not so dangerous...morphia resonance, gods, miracles, paranormal, to the potentially dangerous horoscopes. In Donald Reagan’s book "For the Record” he states that Ronald Reagan’s "....every major decision was cleared with a woman in San Francisco (using) horoscopes..” Now its okay if you believe a horoscope which says you will have a wonderful day and meet someone very interesting BUT if you are moving armies around the world and launching missiles, I would rather that you don’t use a horoscope.
Science is always the most interesting when we can see how ideas develop over time. “Improbability” starts with early randomizing devices like the astragalus and talus used in Egypt, then moves on to early works on probability and chance by Huygens (1657) and Cardano (1563) before tackling the greats: Fermat, Pascal and Bernoulli.
Once the foundation is laid, Hand’s chapter titles are the key to how this may play out without involving the supernatural:
The Law of Inevitability (someone’s got to win the lottery)
The Law of Truly Large Numbers and the law of combinations (the birthday problem)
The Human Mind (our minds are programmed to find patterns and given enough data….)
The Law of Selection (publication bias) and the Law of Near Enough
I love the Law of Near Enough, because I met my wife in Costa Rica in 1987. She lived in Quebec and I lived in Seattle. However, we discovered that two years earlier I was in Cozumel for two weeks at the same time she was in Playa del Carmen only 47 km away. The people I was diving with spent their second week in Playa while I elected to stay behind. Hmmmm, it’s Near Enough that I’ve thought of this many times over the past 25 years. Of course, it could have been 100km, or offset by exactly one month, or one year, or have occurred simultaneously at a completely different place.
When one, looks backwards its easy to find a pattern. Prediction of future events is another matter.
"Understanding means removing obscurity, obfuscation, ambiguity and confusion” David Hand
The Improbability Principle is the counter punch to Carl Jung’s “synchronicity”. Jung, the creator of “archetypes” used in psychotherapy practice, sought to prove that unusual coincidences (or synchronicity) that he documented with his patients must be due to supernatural causes. Unfortunately, Jung was a psychoanalyst not a statistician. David Hand, on the other hand, uses mathematics and statistics to demonstrate that “coincidences” need no extraordinary cause but are simply a clear effect of the law of very large numbers.
Before jumping into the history of probability, which is where this work shines, Hand demonstrates that there is a wide range of beliefs that run the gauntlet from the not so dangerous...morphia resonance, gods, miracles, paranormal, to the potentially dangerous horoscopes. In Donald Reagan’s book "For the Record” he states that Ronald Reagan’s "....every major decision was cleared with a woman in San Francisco (using) horoscopes..” Now its okay if you believe a horoscope which says you will have a wonderful day and meet someone very interesting BUT if you are moving armies around the world and launching missiles, I would rather that you don’t use a horoscope.
Science is always the most interesting when we can see how ideas develop over time. “Improbability” starts with early randomizing devices like the astragalus and talus used in Egypt, then moves on to early works on probability and chance by Huygens (1657) and Cardano (1563) before tackling the greats: Fermat, Pascal and Bernoulli.
Once the foundation is laid, Hand’s chapter titles are the key to how this may play out without involving the supernatural:
The Law of Inevitability (someone’s got to win the lottery)
The Law of Truly Large Numbers and the law of combinations (the birthday problem)
The Human Mind (our minds are programmed to find patterns and given enough data….)
The Law of Selection (publication bias) and the Law of Near Enough
I love the Law of Near Enough, because I met my wife in Costa Rica in 1987. She lived in Quebec and I lived in Seattle. However, we discovered that two years earlier I was in Cozumel for two weeks at the same time she was in Playa del Carmen only 47 km away. The people I was diving with spent their second week in Playa while I elected to stay behind. Hmmmm, it’s Near Enough that I’ve thought of this many times over the past 25 years. Of course, it could have been 100km, or offset by exactly one month, or one year, or have occurred simultaneously at a completely different place.
When one, looks backwards its easy to find a pattern. Prediction of future events is another matter.
December 12, 2019
The Improbability Principle tells about an important concept, albeit in a not so exciting way as it promises.
Basically, the principle stands at that anything that is not going to happen (more so in a never ever way) can and will happen one day and that such instances of such improbable happening are in abundance around us.
Well, sometimes I thought that is common sense and if you look closely at the other laws of the book the situations become more and more an obvious thing.
For in short,the book says that something IS going to happen, with enough trials something of low probability WILL also happen, and that as we look back we see the series of such events together that makes the probability of them happening a much duller event, and that with small changes big reverberations will occur and that anything near enough IS good enough.
The intro does not deserve the book as it fails (thoroughly) to excite the reader.The laws run long into unnecessary examples.The book is filled with gibberish and fillers all over.And the last chapter so name- "How to use the Improbability Principle" fails on the most required promise.
4 stars because the concept is important and no matter how bad the book is executed, the concept comes across and I'll remember it forever (Which as it sounds is not so improbable).
Basically, the principle stands at that anything that is not going to happen (more so in a never ever way) can and will happen one day and that such instances of such improbable happening are in abundance around us.
Well, sometimes I thought that is common sense and if you look closely at the other laws of the book the situations become more and more an obvious thing.
For in short,the book says that something IS going to happen, with enough trials something of low probability WILL also happen, and that as we look back we see the series of such events together that makes the probability of them happening a much duller event, and that with small changes big reverberations will occur and that anything near enough IS good enough.
The intro does not deserve the book as it fails (thoroughly) to excite the reader.The laws run long into unnecessary examples.The book is filled with gibberish and fillers all over.And the last chapter so name- "How to use the Improbability Principle" fails on the most required promise.
4 stars because the concept is important and no matter how bad the book is executed, the concept comes across and I'll remember it forever (Which as it sounds is not so improbable).
February 14, 2015
Statistics. Pretty basic with a nice coherent framework to make it understandable and useful. Examples from everyday life depended a bit heavily on throwing dice. I would have liked to see some poker or other games analyzed and used as examples. I deal with examples of poor statistical thinking every day in my medical office. It is my job to help people understand the true risk of treatment versus symptom or disease - often it is not very clear and information from well meaning relatives and friends and the Internet is often misleading. I also get to see the results of unnecessary testing and procedures - a patient with deformed breasts from multiple unnecessary biopsies from mammograms that should never have been done and the man who almost died from his prostate biopsy. Another old man who died after a lung biopsy for a tumor that would never have hurt him before he died of other causes. Statistics have an importance in real life, but we all know that numbers can be distorted and made to be misleading. The difference between a 1% chance of dying from a procedure and 1.5% chance is a 33.3 percent improvement. That sounds like a lot, but spread it over 10 years and it doesn't mean much, yet medical decisions are made every day on just such analyses. Having a real feel for the meaning of numbers needs to be combined with the statistical analysis. That's what you need a primary care physician for, and this primary care physician is better off for having read this book.
April 11, 2014
I heard an interview on NPR with David Hand in which he shared a seemingly impossible coincidence involving Anthony Hopkins. I retold the story to my wife, who purchased the book for me that same night. I have not been able to shut up about the book ever since. Hand does an incredible job of combining probability theory, psychology, math, history, and truly incredible stories to teach the reader why we should actually expect seemingly unlikely events (I.e. winning the lottery twice in one day) to occur. For someone like myself, who cringes at the words "statistics" or "math," I was amazed at how simple and clear Hand was in leading a reader like myself through mathematical and statistical explanations. If only he had been my graduate school stats professor! In addition to the amazing stories that Hand has included and his clarity in breaking down the probability principles that explain them, Hand is a surprisingly funny writer. I caught myself laughing out loud many times while reading the book (much to the irritation of the people seated near me). Statistician + able to clearly teach sophisticated concepts + interesting + funny! Prior to completing Hand's book, I would have guessed that was an impossible combination.
Not everyone is able to pick up stats or math theories so easily. Here, Hand provides a book accessible to a wide audience that debunks "magic," "luck," "psychic abilities," "divine intervention," or any other number of concepts that signify our inherently poor ability to apply science and reason to everyday events. People in 2014 walk around, drunk on the power of hindsight bias, assuming they are far more intellectually advanced than their ancestors. But the belief in supernatural causes for events that can be explained by science is just as rampant as ever. To some in the stats or math fields this book may seem underwhelming, just as I am not knocked off my seat by psychology books written for the general public. And yet a well-researched, well-written intro level book that debunks common myths using psychology is something I would be the first to applaud. It doesn't appear that Hand's book was intended to be marketed to statisticians- though, being human, one can presume these individuals are as susceptible to being tricked by some strands of the Improbability Principle as the rest of us. Therefore, reviewers who give the book a low rating based simply on the fact that they were already familiar with the theories prior to reading it, might want to reconsider whether the book itself is not good, or whether they simply made a poor selection in choosing to read it.
Not everyone is able to pick up stats or math theories so easily. Here, Hand provides a book accessible to a wide audience that debunks "magic," "luck," "psychic abilities," "divine intervention," or any other number of concepts that signify our inherently poor ability to apply science and reason to everyday events. People in 2014 walk around, drunk on the power of hindsight bias, assuming they are far more intellectually advanced than their ancestors. But the belief in supernatural causes for events that can be explained by science is just as rampant as ever. To some in the stats or math fields this book may seem underwhelming, just as I am not knocked off my seat by psychology books written for the general public. And yet a well-researched, well-written intro level book that debunks common myths using psychology is something I would be the first to applaud. It doesn't appear that Hand's book was intended to be marketed to statisticians- though, being human, one can presume these individuals are as susceptible to being tricked by some strands of the Improbability Principle as the rest of us. Therefore, reviewers who give the book a low rating based simply on the fact that they were already familiar with the theories prior to reading it, might want to reconsider whether the book itself is not good, or whether they simply made a poor selection in choosing to read it.
March 18, 2017
How miracles, rare events and coincidences happen? Moreover, everyday?
Begin this book by asking questions and by the end you may 'probably' reach an answer. It's not however an answer to what causes those rare events but how those events are not actually improbable. It provides a new perspective and a different direction to look at things and their probabilities - how statistics and numbers can alter situations and view points. David Hand tries to explain Improbability through the various strands of his principle - the law of inevitability, law of selection, law of truly large numbers, etc. He explains this through examples (real life and fictional) and gives the book a lively sense. Hand also tries to put up humor to ease the way of the reader throughout the book. Declining the prophecies or stereotypes or superpower - God as the cause behind the extremely improbable event, Hand tries to interpret them through the Improbability Principle.
{Move to Appendices after reading Chapter 2 for better understanding.}
Albeit the Improbability Principle is one of those books which leaves an impact on your mind and has the capacity to alter your perceptions, the book often drifts away from its basic hypothesis. Hand takes a long time to make his point. The last chapter on the applications of the Improbability principle stands too dull compared to the other sections of the book.
Overall the book is very entertaining.
Begin this book by asking questions and by the end you may 'probably' reach an answer. It's not however an answer to what causes those rare events but how those events are not actually improbable. It provides a new perspective and a different direction to look at things and their probabilities - how statistics and numbers can alter situations and view points. David Hand tries to explain Improbability through the various strands of his principle - the law of inevitability, law of selection, law of truly large numbers, etc. He explains this through examples (real life and fictional) and gives the book a lively sense. Hand also tries to put up humor to ease the way of the reader throughout the book. Declining the prophecies or stereotypes or superpower - God as the cause behind the extremely improbable event, Hand tries to interpret them through the Improbability Principle.
{Move to Appendices after reading Chapter 2 for better understanding.}
Albeit the Improbability Principle is one of those books which leaves an impact on your mind and has the capacity to alter your perceptions, the book often drifts away from its basic hypothesis. Hand takes a long time to make his point. The last chapter on the applications of the Improbability principle stands too dull compared to the other sections of the book.
Overall the book is very entertaining.
January 20, 2016
It was fun to reflect on this book during the recent furor over the largest Powerball jackpot to date. And I was especially interested in the meme that circulated for a while claiming that the $1.5 billion could make every American a millionaire. For a brief moment, questions of probability, distributions, and statistics entered the American consciousness. And then, with a winner announced, everyone went back to their regularly scheduled programming and statistical illiteracy remained.
Numbers are hard--they are another language. It is no one's first language. And so this book is an attempt to translate statistics into English and into the everyday context. It is sorely needed since the modern world is built on numbers. Whether we want to or not, we confront statistics and probability every day, yet our evolutionary tendency biases our perceptions of things like large numbers and the actual probability of events. I really liked Hand's book, because he is teaching statistics without teaching statistics. Instead, he is teaching numbers, how to interact with them, what to do when we see them, and how to rework our native thinking about the world we live in.
Read more at http://znovels.blogspot.com/2016/01/t...
March 10, 2019
Come è possibile che capitino tante coincidenze? È vero che c'è un ordine nascosto dietro quello che ci appare casuale? La risposta è negativa, e lo statistico David Hand lo spiega molto bene in questo libro. Hand racconta delle "leggi" che si applicano al nostro pensiero - la legge dell'inevitabilità che afferma che qualcosa deve accadere, la legge dei numeri davvero grandi che dice che se le possibilità sono enormi è facile trovare qualcosa, la legge della selezione che sceglie l'evento a posteriori - che ci fanno vedere coincidenze anche quanto non ci sono. I numerosi esempi che presenta sono di grande aiuto nello spiegare cosa succede, permettendo al lettore di assorbire i concetti: poi magari non se li ricorderà, ma almeno potrà ogni tanto fermarsi a pensare. La traduzione di Andrea Zucchetti è scorrevole, anche se sono rimasti alcuni refusi più tecnici che lessicali. La prefazione di Marco Malvaldi mi è risultata infine un poco incongrua: non certo per il contenuto, quanto per lo stile che è ovviamente malvaldiano. Se volevate prendere il libro a causa sua, forse è meglio che prima guardiate altre pagine (e poi probabilmente lo comprerete lo stesso...)
August 3, 2017
Il titolo è provocatorio, poichè tutti sappiamo che il caso esiste. Il libro tratta in sostanza di teoria delle probabilità, ma senza formule ed in maniera comprensibile a tutti, e con una serie di esempi tratti dalla vita quotidiana o da fatti realmente accaduti. In particolare, come si afferma nella prefazione, ""Questo libro parla di eventi straordinariamente improbabili. Parla delle ragioni per cui accadono cose incredibilmente inverosimili. Di più: parla delle ragioni per cui continuano ad accadere, ancora e ancora e ancora.
L'ho trovato interessante e ben scritto, anche se avrebbe potuto essere più stringato. La breve (per fortuna!) prefazione di Marco Malvaldi è inutile ed abbastanza fuori tema. L'editore furbetto l'ha aggiunta per poter mette in copertina il nome del suddetto autore a scopo incremento vendite. Per questo meriterebbe 1 stella in meno.
L'ho trovato interessante e ben scritto, anche se avrebbe potuto essere più stringato. La breve (per fortuna!) prefazione di Marco Malvaldi è inutile ed abbastanza fuori tema. L'editore furbetto l'ha aggiunta per poter mette in copertina il nome del suddetto autore a scopo incremento vendite. Per questo meriterebbe 1 stella in meno.
June 12, 2017
The Improbability Principle leans on principle, anecdote, and mathematics to underscore what Benjamin Disraeli put best: "there are lies, damn lies, and plenty of reasons to question everything." While there won't be many revelations for the statistically predisposed, the Improbability Principle uses equal parts anecdote and principle to reveal the dirty tricks beneath all sorts of fantastic claims.
Still, repeated examples and an overabundance of detail are to the detriment of a central idea ("if something impossible happens, check your assumptions") that could be well-explored in a much smaller volume. The Improbability Principle is worth understanding, but it won't take a full read to get it.
Still, repeated examples and an overabundance of detail are to the detriment of a central idea ("if something impossible happens, check your assumptions") that could be well-explored in a much smaller volume. The Improbability Principle is worth understanding, but it won't take a full read to get it.
June 2, 2014
Enjoyed it and if you are of similar mind there is a high probability that you will too. I didnt catch a lot of the finer statistical points, but what I really liked was being given names and a handle on the various sub-principles, like the law of inevitability, laws of large and truly large numbers,law of selection, the probability lever, and law of near enough. Ah, if only the airy fairy, ju-ju, woo woo superstitious, reincarnated crystal and navel gazers would read books like this... They just might appreciate the probably improbable more easily and have more joy in reality and life.
September 1, 2016
Overall, I enjoyed this book. There are a lot of laws presented (law of inevitability, law of truly large numbers, etc.) and there were several times when I felt a little overwhelmed by them all. But this offered, to me, a good treatment of the topic of improbability with good examples and without an overabundance of numbers.
June 8, 2014
One of the easier to read and complete unabridged selection on statistics. I couldn't have been any more impressed with the book, well written and covers all the math.
July 4, 2015
Probability.
October 11, 2019
It’s a bit unfair timing for this book that I read it immediately after ‘How Not to Be Wrong’ by Jordan Ellenberg (both were published around the same time). I would have appreciated it a lot more had this reading not been tainted by the inevitable comparison.
Content: The improbability principle focused on using probability theory to explain phenomena that are often constructed as improbable, or even (intentionally) interpreted as supernaturally inspired. David J. Hand uses approachable lay-person languages to explain these phenomena (such as: law of truly large numbers, look elsewhere effect), which are valuable names for a general audience to recognize the 'improbability' theme across different domains. However, most topics and examples (lottery, car accidents, diseases, ancient script decoding, etc) are covered by the probability-related chapters in ‘How Not to Be Wrong’. One topic not extensively covered in ‘How Not to Be Wrong’ is the last chapter about the anthropic principle, which reads like a Richard Dawkins.
Writing & Audio-book Narration: In addition to the unfortunate timing that most examples reads familiar (having just encountered them in ‘How Not to Be Wrong’), both Hand’s writing and Paul Hodgson narration are detectible English, elevated, composed, reserved(clinical), and rarely emotionally roaring like Ellenberg. I am listening to this audiobook with low-cognitive tasks (driving, exercising, chores), and it was a bit easy to lose focus or train of thoughts.
It would be better to appreciate this book if I return to it after a hiatus from math/probability reading. But I so much need the nerve-calming from math/probability reading in this chaotic time. As Ellenberg says about math in his book: it puts you in direct contact with the universe, it’s here before you, and it would be here after you. You will get the nerve-calming math ideas in ‘The Improbability Principle’, but don’t expect poetic proclamations like those in ‘How Not to Be Wrong’.
Content: The improbability principle focused on using probability theory to explain phenomena that are often constructed as improbable, or even (intentionally) interpreted as supernaturally inspired. David J. Hand uses approachable lay-person languages to explain these phenomena (such as: law of truly large numbers, look elsewhere effect), which are valuable names for a general audience to recognize the 'improbability' theme across different domains. However, most topics and examples (lottery, car accidents, diseases, ancient script decoding, etc) are covered by the probability-related chapters in ‘How Not to Be Wrong’. One topic not extensively covered in ‘How Not to Be Wrong’ is the last chapter about the anthropic principle, which reads like a Richard Dawkins.
Writing & Audio-book Narration: In addition to the unfortunate timing that most examples reads familiar (having just encountered them in ‘How Not to Be Wrong’), both Hand’s writing and Paul Hodgson narration are detectible English, elevated, composed, reserved(clinical), and rarely emotionally roaring like Ellenberg. I am listening to this audiobook with low-cognitive tasks (driving, exercising, chores), and it was a bit easy to lose focus or train of thoughts.
It would be better to appreciate this book if I return to it after a hiatus from math/probability reading. But I so much need the nerve-calming from math/probability reading in this chaotic time. As Ellenberg says about math in his book: it puts you in direct contact with the universe, it’s here before you, and it would be here after you. You will get the nerve-calming math ideas in ‘The Improbability Principle’, but don’t expect poetic proclamations like those in ‘How Not to Be Wrong’.
March 19, 2022
An interesting read on statistics. I liked the way how the many strands of the Improbability Principle were explained, using examples from history or daily life, and how this makes you realize that events that shouldn't happen (that is, events that have a very small chance of occurring) do happen.
A couple of weeks ago I was arguing with a friend about Monty Hall's problem. She couldn't understand why your chances of winning a car are actually higher if you change your first choice. Likewise, many of the improbabilities seem counterintuitive at first, but when you look at the bigger picture, you start to understand the logic behind the occurrence of the improbable events.
A couple of weeks ago I was arguing with a friend about Monty Hall's problem. She couldn't understand why your chances of winning a car are actually higher if you change your first choice. Likewise, many of the improbabilities seem counterintuitive at first, but when you look at the bigger picture, you start to understand the logic behind the occurrence of the improbable events.
November 21, 2024
I recently revisited David Hand's _The Improbability Principle_ after a three-year hiatus, and I was struck by how much I had forgotten. This book offers a fascinating exploration of chance and coincidence, introducing fundamental statistical concepts in an accessible way. Through five key laws, he reveals the surprising patterns that emerge from seemingly random events.
While the book provides a solid foundation in probability and statistics, I'm still working on more practical applications with intentional practice, reflection, and integration into my decision-making process.
While the book provides a solid foundation in probability and statistics, I'm still working on more practical applications with intentional practice, reflection, and integration into my decision-making process.
July 6, 2014
The title of mathematics professor David Hand’s book “The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day” seems like a contradiction. But it is not. Events that would be improbable for one or two people over a short time will become probable, indeed almost certain, when involving a large number of people and a huge amount of time. Hand calls this the law of truly large numbers, “which says that, with a large enough number of opportunities, any outrageous thing is likely to happen.”
Hand uses the example of a woman who won the New Jersey Lottery twice in four months, the first time in 1985 and again the next year, winning $5.4 million in total. The probability that one person would win a state lottery twice in four months is exceedingly low. But such an event becomes likely when tens of millions of people play various lotteries week after week over decades.
An example I would use to demonstrate the law of truly large numbers is the likelihood of there being other planets with intelligent life. Looking just at Earth, we see that many factors had to coincide to allow the emergence of life, and later, intelligent life on this planet. Our Sun had to be the right size and brightness; Earth’s orbit had to be in the “habitable zone,” allowing surface temperatures neither too cold or too hot; Earth had to develop the right kind of atmosphere and a magnetic field to shield the surface from harmful radiation; and many other things. Needing all these factors to come together at the same place made many scientists believe that intelligent life in the universe might be rare and, at the extreme, that we were the only intelligent life in the universe.
But recent discoveries by earth-bound and orbiting telescopes and extrapolation show us that there may be 100-200 billion stars in our galaxy alone. Scientists estimate that there are 100-200 billion galaxies in the observable universe, meaning there may be 100 sextillion stars in our observable universe. Many if not most of them will have planets. The likelihood that there are a 100 billion planets in our own galaxy and sextillions more in other galaxies makes it probable, to my mind, that there is abundant life spread through the cosmos.
This 2014 book is a good introduction to probability. Even better books on the subject, in my opinion, would be “The Drunkard’s Walk” by Leonard Mlodinow and “The Black Swan” by Nassim Nicholas Taleb. My only complaint about Hand’s volume is there is a lot of repetition of the central thesis. But I will say that Hand gives a detailed look at different aspects of probability and says interesting things about mathematics, religion, and science.
“Pure mathematics yields absolute truth because it is simply the deduction of the consequences which follow from a given set of axioms when you apply a given set of rules. This means that in pure mathematics you define your own universe, so that you can certainly state the absolute truth within it. And religion as an expression of faith is a statement of belief in an absolute truth.
“In contrast, science is all about possibilities. We propose theories, conjectures, hypotheses, and explanations. We collect evidence and data, and we test the theories against this new evidence. If the data contradict our theory, then we change the theory. In this way science advances, and we gain greater and greater understanding. … It's the very essence of science that its conclusions can change, that is, that its truths are not absolute. The intrinsic good sense of this is contained within the remark reportedly made by the eminent economist John Maynard Keynes, responding to the criticism that he had changed his position on monetary policy during the 1930s Depression: ‘When the facts change, I change my mind. What do you do, sir?’”
I always appreciate it when the writer on probability delves more deeply through the mathematics and heads into philosophy. Hand doesn’t do this often in this book, but he did state: “The uncertain and unpredictable are central to the mystery of human existence and to our attempts to understand the universe.”
Considering that we humans know a microscopic fraction of what there is to know in just our immediate surroundings, surviving and thriving on Earth is crap shoot. Human life is beset by surprises, occasionally deadly. We don’t know when one of our bodily processes will break down. We don’t know if or when we will be hit by crime, financial setback, unemployment, or be confronted by a neighbor with mental problems who owns a cache of weapons.
Our only consolation in the 21st century is that we have the capabilities, through our science, for understanding the forces of nature, discerning patterns in life, and being able to gauge probabilities and thus make predictions. We can battle deadly microbes, repair our bodies, prepare for floods, tornadoes, hurricanes, and other disasters. We can know if our picnic in two days will likely have a sunny day.
As opposed to our ancestors of 150 or more years ago, who did not have our science or much of a fighting chance against the world’s vicissitudes and disasters, we in the 20th and 21st centuries – especially in the First World – know more of our universe and its laws, can calculate life’s probabilities, and live with fewer existential anxieties.
Hand uses the example of a woman who won the New Jersey Lottery twice in four months, the first time in 1985 and again the next year, winning $5.4 million in total. The probability that one person would win a state lottery twice in four months is exceedingly low. But such an event becomes likely when tens of millions of people play various lotteries week after week over decades.
An example I would use to demonstrate the law of truly large numbers is the likelihood of there being other planets with intelligent life. Looking just at Earth, we see that many factors had to coincide to allow the emergence of life, and later, intelligent life on this planet. Our Sun had to be the right size and brightness; Earth’s orbit had to be in the “habitable zone,” allowing surface temperatures neither too cold or too hot; Earth had to develop the right kind of atmosphere and a magnetic field to shield the surface from harmful radiation; and many other things. Needing all these factors to come together at the same place made many scientists believe that intelligent life in the universe might be rare and, at the extreme, that we were the only intelligent life in the universe.
But recent discoveries by earth-bound and orbiting telescopes and extrapolation show us that there may be 100-200 billion stars in our galaxy alone. Scientists estimate that there are 100-200 billion galaxies in the observable universe, meaning there may be 100 sextillion stars in our observable universe. Many if not most of them will have planets. The likelihood that there are a 100 billion planets in our own galaxy and sextillions more in other galaxies makes it probable, to my mind, that there is abundant life spread through the cosmos.
This 2014 book is a good introduction to probability. Even better books on the subject, in my opinion, would be “The Drunkard’s Walk” by Leonard Mlodinow and “The Black Swan” by Nassim Nicholas Taleb. My only complaint about Hand’s volume is there is a lot of repetition of the central thesis. But I will say that Hand gives a detailed look at different aspects of probability and says interesting things about mathematics, religion, and science.
“Pure mathematics yields absolute truth because it is simply the deduction of the consequences which follow from a given set of axioms when you apply a given set of rules. This means that in pure mathematics you define your own universe, so that you can certainly state the absolute truth within it. And religion as an expression of faith is a statement of belief in an absolute truth.
“In contrast, science is all about possibilities. We propose theories, conjectures, hypotheses, and explanations. We collect evidence and data, and we test the theories against this new evidence. If the data contradict our theory, then we change the theory. In this way science advances, and we gain greater and greater understanding. … It's the very essence of science that its conclusions can change, that is, that its truths are not absolute. The intrinsic good sense of this is contained within the remark reportedly made by the eminent economist John Maynard Keynes, responding to the criticism that he had changed his position on monetary policy during the 1930s Depression: ‘When the facts change, I change my mind. What do you do, sir?’”
I always appreciate it when the writer on probability delves more deeply through the mathematics and heads into philosophy. Hand doesn’t do this often in this book, but he did state: “The uncertain and unpredictable are central to the mystery of human existence and to our attempts to understand the universe.”
Considering that we humans know a microscopic fraction of what there is to know in just our immediate surroundings, surviving and thriving on Earth is crap shoot. Human life is beset by surprises, occasionally deadly. We don’t know when one of our bodily processes will break down. We don’t know if or when we will be hit by crime, financial setback, unemployment, or be confronted by a neighbor with mental problems who owns a cache of weapons.
Our only consolation in the 21st century is that we have the capabilities, through our science, for understanding the forces of nature, discerning patterns in life, and being able to gauge probabilities and thus make predictions. We can battle deadly microbes, repair our bodies, prepare for floods, tornadoes, hurricanes, and other disasters. We can know if our picnic in two days will likely have a sunny day.
As opposed to our ancestors of 150 or more years ago, who did not have our science or much of a fighting chance against the world’s vicissitudes and disasters, we in the 20th and 21st centuries – especially in the First World – know more of our universe and its laws, can calculate life’s probabilities, and live with fewer existential anxieties.
November 7, 2017
I love mathematics. I love statistics. Above all I love logical reasoning being applied to answer questions. But even I struggled to get through this book. Whilst the content definitely has an element of appeal, I did find it dry more often than not. If you are considering it, do yourself a favour and go for the Audible audio book version.
November 22, 2024
Once I swept the cobb webs from my brain I realized how great this book is. This is a great read for those who aren't employed in the fields of mathematics or statistics .
October 30, 2017
Good for critical thinking training.
August 12, 2016
On April 23, 1999, Fernando Tatis of the St. Louis Cardinals hit a grand slam home run against Chan Ho Park of the Dodgers. That's not an overly rare event. However, Tatis didn't hit just one grand slam off of Park. He hit TWO. And they were in the same inning. No Major League player had ever done this before and no one has done it since. The chances of being a witness to such a thing must be so high to make it unlikely that anyone would ever see it. And yet it happened.
David Hand's book "The Improbability Principle" tries to explain how such seemingly unlikely events not only do happen, they almost have to happen. Hand's "principle" is actually just his way of synthesizing several different mathematical laws of probabilities. He does an excellent job of explaining complicated topics in a clear and engaging manner.
One of the most important laws to take from the book is The Law of Truly Large Numbers. Essentially, if you give some event enough chances to occur, it will happen. So if you think someone winning the lottery twice is impossible, it isn't. People have done it. And with big jackpots. Why? Because there are a lot of lotteries in the world. There are so many that by chance one person will take home two big payouts.
There is also the Law of Near Enough. Many things that humans think are coincidences are just our way of drawing parallels between things that are close. Say that I know the value of pi to 20 digits. You know it to 10 digits. We both think we know the value of pi. But we don't both know the exact same number. But for most people it's near enough.
Hand also mentions probability levers. Certain outside influences can change the probability of an event occurring. In the case of Tatis' two grand slams, he was playing in an era when there were many home runs hit. He was also facing the same pitcher twice and he was tiring as he was throwing many more pitches than normal. Both of those factors made the two grand slam home runs a bit more likely, albeit still highly unusual.
Our brains are trained to look for patterns and coincidences it seems. But our brains aren't as well trained at figuring out probabilities. We go to casinos and play roulette and bet on numbers that haven't come up recently because "they're due." Basketball players keep passing the ball to a teammate who may have made two shots in a row because "he has a hot hand." But, these can all just be random variations. It just happens.
The Improbability Principle may make you think that everything is random. Weird coincidences may seem a little less special, but you may also feel special knowing the reason why that is so.
Reviewed by Bob Timmermann, Senior Librarian, Science, Technology & Patents Dept
David Hand's book "The Improbability Principle" tries to explain how such seemingly unlikely events not only do happen, they almost have to happen. Hand's "principle" is actually just his way of synthesizing several different mathematical laws of probabilities. He does an excellent job of explaining complicated topics in a clear and engaging manner.
One of the most important laws to take from the book is The Law of Truly Large Numbers. Essentially, if you give some event enough chances to occur, it will happen. So if you think someone winning the lottery twice is impossible, it isn't. People have done it. And with big jackpots. Why? Because there are a lot of lotteries in the world. There are so many that by chance one person will take home two big payouts.
There is also the Law of Near Enough. Many things that humans think are coincidences are just our way of drawing parallels between things that are close. Say that I know the value of pi to 20 digits. You know it to 10 digits. We both think we know the value of pi. But we don't both know the exact same number. But for most people it's near enough.
Hand also mentions probability levers. Certain outside influences can change the probability of an event occurring. In the case of Tatis' two grand slams, he was playing in an era when there were many home runs hit. He was also facing the same pitcher twice and he was tiring as he was throwing many more pitches than normal. Both of those factors made the two grand slam home runs a bit more likely, albeit still highly unusual.
Our brains are trained to look for patterns and coincidences it seems. But our brains aren't as well trained at figuring out probabilities. We go to casinos and play roulette and bet on numbers that haven't come up recently because "they're due." Basketball players keep passing the ball to a teammate who may have made two shots in a row because "he has a hot hand." But, these can all just be random variations. It just happens.
The Improbability Principle may make you think that everything is random. Weird coincidences may seem a little less special, but you may also feel special knowing the reason why that is so.
Reviewed by Bob Timmermann, Senior Librarian, Science, Technology & Patents Dept
September 3, 2017
Il caso non esiste, ma esiste la probabilità
A rendere il testo ancora più indigesto di quanto lo sia di per sé, è l'introduzione del tutto superflua e a malapena sopportabile di Malvaldi (non si capisce come ci entri in questo discorso, se non per mettereci una firma con un registro linguistico puerile e del tutto inadeguato al contesto e al registro linguistico di Hand - il primo scrive gialli per i pecoroni italiani, il secondo è un matematico con una cattedra all'Imperial College di Londra). Il professor Hand è noioso, come tutti quelli che si occupano di statistica. In particolare cerca in ogni modo e in ogni pagina di convincere il lettore che qualunque teoria su caso e destino (o qualunque altra cosa tenti di spiegare "l'inspiegabile") sia in realtà o frutto della fallacità della mente umana (a riguardo ci sono tonnellate di testi) o frutto del caso. In particolare non si fa scrupolo di citare Jung e distorcere il senso della "sincronicità", a volte prendendo esempi di sogni che in realtà si spiegherebbero meglio con la teoria di T.C.Lethbridge (altro accademico inglese bandito dalla comunità). Come non si fa scrupolo di citare alla lettera la teoria evoluzionistica di Darwin, quando ormai molti biologi, benché siano consci che sia la strada corretta, abbiano forti riserve sul fatto che sia perfetta in sé (e vorrei ben vedere). Il resto sono una serie di esempi banali che rivelano l'ovvio: siamo in 7 miliardi di persone, come non possono verificarsi centinaia di morti per fulmine all'anno? Suvvia! E' ovvio che è tutta questione di probabilità. E alla fine si passa alla fisica e alla matematica (per fortuna il professore ridicolizza anche quella gigantesca stronzata del "Monster" e il "Moonshine"), e là chiaramente non si sbaglia mai. In conclusione lo trovo un testo parecchio molesto, come del resto lo sono gli statistici.
A rendere il testo ancora più indigesto di quanto lo sia di per sé, è l'introduzione del tutto superflua e a malapena sopportabile di Malvaldi (non si capisce come ci entri in questo discorso, se non per mettereci una firma con un registro linguistico puerile e del tutto inadeguato al contesto e al registro linguistico di Hand - il primo scrive gialli per i pecoroni italiani, il secondo è un matematico con una cattedra all'Imperial College di Londra). Il professor Hand è noioso, come tutti quelli che si occupano di statistica. In particolare cerca in ogni modo e in ogni pagina di convincere il lettore che qualunque teoria su caso e destino (o qualunque altra cosa tenti di spiegare "l'inspiegabile") sia in realtà o frutto della fallacità della mente umana (a riguardo ci sono tonnellate di testi) o frutto del caso. In particolare non si fa scrupolo di citare Jung e distorcere il senso della "sincronicità", a volte prendendo esempi di sogni che in realtà si spiegherebbero meglio con la teoria di T.C.Lethbridge (altro accademico inglese bandito dalla comunità). Come non si fa scrupolo di citare alla lettera la teoria evoluzionistica di Darwin, quando ormai molti biologi, benché siano consci che sia la strada corretta, abbiano forti riserve sul fatto che sia perfetta in sé (e vorrei ben vedere). Il resto sono una serie di esempi banali che rivelano l'ovvio: siamo in 7 miliardi di persone, come non possono verificarsi centinaia di morti per fulmine all'anno? Suvvia! E' ovvio che è tutta questione di probabilità. E alla fine si passa alla fisica e alla matematica (per fortuna il professore ridicolizza anche quella gigantesca stronzata del "Monster" e il "Moonshine"), e là chiaramente non si sbaglia mai. In conclusione lo trovo un testo parecchio molesto, come del resto lo sono gli statistici.
June 28, 2017
هناك قوانين للصدف تفسر لماذا تحدث الاشياء المستبعد حدوثها ، و نظرا لان عمل هذه القوانين يتم بشكل معاكس لإدراكنا الفطري للعالم، فنحن نميل الى اعتبار هذه الاشياء معجزات.
ففي عام ١٩٧٢، عندما وقع الممثل انتوني هوبكنز عقد فيلم مبني على رواية فتاة من بتروفوكا ل جي فيفر ، سافر الى لندن لشراء نسخة من الرواية لدراسة دوره ، و لكن لسوء الحظ ، لم توجد الرواية في كافة المكتبات الشهيرة اليت ذهب اليها هوبكنز.
و في طريق عودته الى منزله بالمترو وجد كتابا على المقعد المجاور له . الغريب ان يكون هذا الكتاب هو رواية فتاة من بتروفكا. عندما قابل هوبكنز فيفر المؤلف و اخبره عن النسخة فتبين ان هذه النسخة هي نسخة المؤلف نفسه و التي كان قد اعارها لأحد أصدقاءه، و التي أضاعها بدوره من فترة.
و يفسر دكتور هاند هذه الأحداث من خلال مبدا " الأحداث غير المُحتملة" ، و الذي يتكون من خمس قوانين متشابكة، و هي قانون الحتمية، و قانون الاعداد الكبيرة فعلا، و قوانين رافعة الاحتمالات ، و قانون القريب يكفي.
و اروع ما في الكتاب هو تجنيب القاريء العديد من التفاصيل الأخصائية و الرياضية ..
كتاب جميل ادعو لقرائته
و تجدون مراجعة اكبر الكتاب على صفحتي على الفايس بوك .
ففي عام ١٩٧٢، عندما وقع الممثل انتوني هوبكنز عقد فيلم مبني على رواية فتاة من بتروفوكا ل جي فيفر ، سافر الى لندن لشراء نسخة من الرواية لدراسة دوره ، و لكن لسوء الحظ ، لم توجد الرواية في كافة المكتبات الشهيرة اليت ذهب اليها هوبكنز.
و في طريق عودته الى منزله بالمترو وجد كتابا على المقعد المجاور له . الغريب ان يكون هذا الكتاب هو رواية فتاة من بتروفكا. عندما قابل هوبكنز فيفر المؤلف و اخبره عن النسخة فتبين ان هذه النسخة هي نسخة المؤلف نفسه و التي كان قد اعارها لأحد أصدقاءه، و التي أضاعها بدوره من فترة.
و يفسر دكتور هاند هذه الأحداث من خلال مبدا " الأحداث غير المُحتملة" ، و الذي يتكون من خمس قوانين متشابكة، و هي قانون الحتمية، و قانون الاعداد الكبيرة فعلا، و قوانين رافعة الاحتمالات ، و قانون القريب يكفي.
و اروع ما في الكتاب هو تجنيب القاريء العديد من التفاصيل الأخصائية و الرياضية ..
كتاب جميل ادعو لقرائته
و تجدون مراجعة اكبر الكتاب على صفحتي على الفايس بوك .
May 6, 2014
Whole lotta statistics going on here. This book didn't shy away from the statistical thinking that buttresses the title, but it was done in a manner that kept it both readable and interesting. If you're a fan of superstition or the supernatural this book will ruin your party. If, however, you're interested in finding out how and why some of those things can, do and should occur, dig in, this book is built for you.
July 23, 2014
Good discussion of the reasons that human beings are terrible at estimating probability. I especially enjoyed the discussion showing that many of the events we hear about that shouldn't have happened because the probability of their occurrence was so small really indicate that we've assigned the events the wrong probability. The author illustrates this with examples from finance and from astronomy.
September 26, 2014
Enjoyed how Hand presents complicated information. The ideas around probabilities and understanding that even “miraculous” events can be explained scientifically broaden my views. Considering I prefer to know the why’s of life, it is good to be educated on the how’s. Seeing the world from different angles makes my understanding richer. This is a fairly technical read – and while it is presented in layman’s terms, not something that is zipped through.
August 16, 2014
This is a very good book that is successful in conveying the mathematical basis for for why rare events are rare but still happen all of the time. The equations are the book's strongest point and its weakness lies with its application of theory. The basis of improbability in history is thin and derived from numerous earlier works that did a better job of explaining these ideas in-depth. If you're looking for hard numbers or an introduction to the topic, this is a good place to start.
Displaying 1 - 30 of 198 reviews