The Art of Statistics: Learning from Data

by David Spiegelhalter

This value of 3/7 may seem odd, as the intuitive estimate might be 2/5 – the proportion of red balls landing to the left of the line. Instead Bayes showed that in these circumstances we should estimate the position as This means, for example, that before any red balls are thrown at all, we can estimate the position to be (0 + 1)/(0 + 2) = ½, whereas the intuitive approach might suggest that we could not give any answer since there is not yet any data. Essentially Bayes is making use of the information about how the position of the line has been initially decided, since we know it is picked at random by throwing the white ball. This initial information takes the same role as the prevalence used in breast screening or dope-testing – it is known as prior information and it influences our final conclusions. In fact, since Bayes’ formula adds one to the number of red balls to the left of the line and two to the total number of red balls, we might think of it as being equivalent to having already thrown two ‘imaginary’ red balls, and one having landed each side of the dashed line. Note that if none of the five balls had landed to the left of the dashed line, we would not have estimated its position as 0/5, but instead as 1/7, which seems much more sensible. The Bayes’ estimate can never be 0 or 1, and is always nearer to ½ than the simple proportion: this is known as shrinkage, in that estimates are always pulled in or shrunk, towards the centre of the initial distribution, in t...

Bayesian application