Statistical Rethinking: A Bayesian Course with Examples in R and Stan (Chapman & Hall/CRC Texts in Statistical Science Book 122)

by Richard McElreath

The average likelihood, Pr(w), can be confusing. It is commonly called the “evidence” or the “probability of the data,” neither of which is a transparent name. The probability Pr(w) is merely the average likelihood of the data. Averaged over what? Averaged over the prior. It’s job is to standardize the posterior, to ensure it sums (integrates) to one. In mathematical form:

The operator E means to take an expectation. Such averages are commonly called marginals in mathematical statistics, and so you may also see this same probability called a marginal likelihood.

The average likelihood on the bottom just standardizes the counts...

theorem. In this book, you’ll meet three different conditioning engines, numerical techniques for computing posterior distributions: (1) Grid approximation (2) Quadratic approximation (3) Markov chain Monte Carlo (MCMC)

In even moderately complex problems, however, the details of fitting the model to data force us to recognize that our numerical technique influences our inferences.

This is because different mistakes and compromises arise under different techniques. The

(3) The probability that it is Monday, given that it is raining.