The Art of Statistics: How to Learn from Data

by David Spiegelhalter

We over-fit when we go too far in adapting to local circumstances, in a worthy but misguided effort to be ‘unbiased’ and take into account all the available information.