Goodreads helps you keep track of books you want to read.
Start by marking “Information Theory, Inference and Learning Algorithms” as Want to Read:
Information Theory, Inference and Learning Algorithms
Enlarge cover
Rate this book
Clear rating
Open Preview

Information Theory, Inference and Learning Algorithms

4.48  ·  Rating details ·  378 ratings  ·  21 reviews
Information theory and inference, often taught separately, are here united in one entertaining textbook. These topics lie at the heart of many exciting areas of contemporary science and engineering - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics, and cryptography. This textbook introduces th ...more
Hardcover, 640 pages
Published October 6th 2003 by Cambridge University Press (first published June 15th 2002)
More Details... Edit Details

Friend Reviews

To see what your friends thought of this book, please sign up.

Reader Q&A

Be the first to ask a question about Information Theory, Inference and Learning Algorithms

Community Reviews

Showing 1-30
Average rating 4.48  · 
Rating details
 ·  378 ratings  ·  21 reviews

More filters
Sort order
Start your review of Information Theory, Inference and Learning Algorithms
Jon Gauthier
NB: Both book and lectures are available for free online. (Check YouTube for lectures.)
Brian Powell
I've had a long and fruitful relationship with this text. It's been with me through several career shifts and has satisfied various, random fits of curiosity. I was introduced to this book in grad school while trying to use computational methods of Bayesian inference to study the early universe (specifically, MCMC, Bayesian model selection, and other sampling techniques). MacKay's coverage of this material is both conceptually clear and practically-minded, and helped me a great deal. Much of the ...more
Dec 13, 2008 added it
Shelves: prob-n-stat
Hokey the Bayesian Bear says: "Only you can prevent the misguided use of p-values."
Jul 23, 2008 rated it it was amazing
This book is amazing! Its a pretty esoteric approach to teaching machine learning and I don't think its a good introductory book on that subject. But for folks already versed in the topic, this book can shed a lot of new light and does a good job abstracting it with concepts from information theory and stats.

This book was my first in depth exposure to information theory and the proofs, often accompanied by helpful figures, were clear and, hell, even exciting. Its a much easier read than Cover &
Jul 27, 2019 rated it it was amazing
Brilliantly exposited. An important read for anyone interested in these topics.
Jul 01, 2008 rated it liked it
Shelves: reference
I chose this to accompany my reading of Norvig's text on artificial intelligence. I thought the information theoretic concepts deepened my understanding of intelligent agents functioning in an information-deprived environment. The sections on genetic algorithms and neural networks gave a nifty information theoretic perspective on those topics, but I think other texts (such as Koza on genetic algorithms) were better reads.

I shall add this to my "reference" collection, for I find myself returning
May 28, 2013 rated it it was amazing
One of the best introductions to information theory, coding (lossy and lossless) and Bayesian approaches to decoding and to inference. This firmly grounds machine learning algorithms in a Bayesian paradigm and gives people the intuition for the subject. The problem sections are not just great, they are absolutely worth doing.
May 22, 2011 rated it it was amazing
Shelves: phd-starter-kit
Excellent book about diverse topics in machine learning, statistics, information theory etc. Many exercises and applications.
Free to download on the internet!
May 30, 2020 rated it it was amazing
Unbelievably clear thinker. I just wish I had the logical stamina to follow his arguments. Alas the maths undergrad me would be so disappointed.
Jethro Kuan
Jul 05, 2018 rated it it was amazing
Excellently written, would revisit again.
Marek Barak
Sep 20, 2018 rated it it was amazing
If you are looking for a simple introduction to Bayesian machine learning, this book is a perfect fit.
Kent Sibilev
Dec 02, 2019 rated it it was amazing
Amazing treatment of the information theory and the Bayesian inference in general.
Sep 27, 2016 rated it it was amazing
Recommends it for: scientists/engineers
I really enjoy(ed) working with this book.
The (>400) problems are interesting, the writing clever and motivational.
While deliberating buying the book, I came across many reviews giving the impression that this was an upper-tier book meant only for those already well-versed in bayesian inference, information theory, and machine learning. Fortunately for me (having purchased it for ~50$), I have been gliding along at quite an easy pace. Already I've learnt about hamming codes and the formulas & axioms (interestingly formulated!) of bayesian probability theory. The treatment probably isn't the most sophisticate ...more
Jimmy Longley
Nov 04, 2016 rated it it was amazing
Shelves: textbooks
Reviewed as part of my 100 books challenge:

Run-on Sentence Summary

A fresh and entertaining textbook that walks through the fundamentals of information theory and machine learning.


Mackay’s prose is fast paced but lucid, and perfect for a self learner. Often when reading CS textbooks, I’ll skim over problems because I can’t be bothered to spin up whatever boilerplate they want me to download off of the website, but this book did a great job of highlig
Tarun Thammisetty
Aug 24, 2016 rated it it was amazing
One of the very rare academic texts which balances intuition and mathematical rigour. The way the author establishes the relationship between Information theory, Inference and Learning is exceptional. An absolute joy to read.
Apr 23, 2019 rated it it was amazing
An exceptional read which gave me so much more confidence in statistics for data science. Fantastic relatable real world questions make this book an absolute classic. Have also read pattern recognition and machine learning, which is also recommend and foundations of data science which isn't as good
Dec 24, 2010 rated it really liked it
Shelves: computer-science
A review of information theory, coding theory, and several machine learning and statistics topics, all from a Bayesian perspective. Low-density parity-check codes (which are used in HDTV) are very cool!
RJ Skerry-Ryan
Jul 02, 2011 rated it it was amazing
I've been working through this chapter by chapter for about a month now. Loving it sofar!
Bolin Zhou
rated it liked it
Dec 02, 2017
Abdullah Güneydaş
rated it it was amazing
Jul 16, 2014
Vincent Warmerdam
rated it it was amazing
Apr 21, 2016
rated it it was amazing
Jan 29, 2012
Mark Jajeh
rated it it was amazing
Apr 17, 2019
rated it it was amazing
Sep 22, 2017
Dimitrij Pinneker
rated it it was amazing
Oct 31, 2014
Hanamanth Rathod
rated it really liked it
Dec 11, 2017
박은정 Park
rated it liked it
Dec 01, 2012
« previous 1 3 4 5 6 7 8 9 next »
There are no discussion topics on this book yet. Be the first to start one »

Readers also enjoyed

  • Pattern Recognition and Machine Learning
  • The Elements of Statistical Learning: Data Mining, Inference, and Prediction
  • Deep Learning
  • Artificial Intelligence: A Modern Approach
  • Bayesian Data Analysis
  • Introduction to Algorithms
  • Machine Learning: A Probabilistic Perspective
  • Pattern Classification
  • Convex Optimization
  • Linear Algebra Done Right
  • An Introduction to Statistical Learning: With Applications in R
  • Philosophical Investigations
  • The Wages of Destruction: The Making and Breaking of the Nazi Economy
  • The Worldly Philosophers
  • Inverting the Pyramid: The History of Football Tactics
  • Science Is Not Enough: Reflections on the Present and Future
  • Lost Victories: The War Memoirs of Hitler's Most Brilliant General
  • The Book of Why: The New Science of Cause and Effect
See similar books…

Goodreads is hiring!

If you like books and love to build cool products, we may be looking for you.
Learn more »
David MacKay was a Professor in the Department of Physics at the University of Cambridge. He studied Natural Sciences at Cambridge and then obtained his PhD in Computation and Neural Systems at the California Institute of Technology. He returned to Cambridge as a Royal Society research fellow at Darwin College. He was internationally known for his research in machine learning, information theory, ...more

News & Interviews

Need another excuse to treat yourself to a new book this week? We've got you covered with the buzziest new releases of the day. To create our...
6 likes · 6 comments