Tien Le-Quang’s Reviews > Information Theory, Inference, and Learning Algorithms > Status Update

Lempel-Ziv coding

Note from Mackay: given any ergodic source (one that is memoryless on sufficiently long timescales), the Lempel-Ziv algorithm can be proven asymptotically to compress down to the entropy of the source.

…and… a general purpose compressor that can discover the probability distribution of any source would be a general purpose artificial intelligence. (not yet exist)