Graphical Models, Exponential Families, and Variational Inference (Foundations and Trends

by Martin J. Wainwright, Michael I Jordan

The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances-including the key problems of computing marginals and modes of probability distributions-are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, Graphical Models, Exponential Families and Variational Inference develops general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. It describes how a wide variety of algorithms- among them sum-product, cluster variational methods, expectation-propagation, mean field methods, and max-product-can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

Genres: Mathematics, Textbooks

324 pages, Paperback

First published December 16, 2008

Reviews

[Suman · Rating 4 out of 5]

A better but less informative title for this book would be "A Heartbreaking Work of Staggering Genius", but since that's a Dave Eggers book, I guess the current title will have to do.

WARNING: Nerd-speak ahead.

Known as "the monograph" in my department, this is a rather long survey paper outlining how certain "unprincipled" approaches, such as Loopy Belief Propagation, Expectation-Propagation, and Mean Field Methods, can be reduced to convex optimization problems when restricting models to exponential families. The perspective is rather interesting for those who like the theoretical approach, but I do wish there were exercises to let the information sink in. I will probably read it again at some point.

I also don't understand why this was published as a book - the paper is free (and legal) online.