Thus one purpose of this presentation of information theory, besides its usefulness, is to sensitize you to this danger, or, if you prefer, to how to use it to get what you want! It has long been recognized the initial definitions determine what you find, much more than most people care to believe. The initial definitions need your careful attention in any new situation, and they are worth reviewing in fields in which you have long worked so you can understand the extent to which the results are a tautology and not real results at all. There is the famous story by Eddington about some people
...more
