Information Geometry (Part 20)

Last time we worked out an analogy between classical mechanics, thermodynamics and probability theory. The latter two look suspiciously similar:

 Classical Mechanics  Thermodynamics  Probability Theory  q  positionextensive variablesprobabilities p  momentumintensive variablessurprisals S   actionentropy Shannon entropy

This is no coincidence. After all, in the subject of statistical mechanics we explain classical thermodynamics using probability theory—and entropy...