Bayes' theorem for a posterior distribution
Statement
If has prior density and data has likelihood , then the posterior density of given is , where ; equivalently .
Why is it true?
This is nothing more than the definition of a conditional density applied to and jointly: the posterior is the joint density of divided by the marginal density of alone, and the joint density factors as prior times likelihood.
Proof sketch
The joint density of factors two ways: as (prior times likelihood, by the definition of the likelihood as the conditional density of given ) and as (posterior times the marginal density of , by the definition of the posterior as the conditional density of given ). Since both expressions equal the same joint density, .
Solving for gives , provided .
It remains to check is exactly the right normalizing constant: integrating both sides of the factorization over , the left side becomes by definition, and the right side becomes since is a probability density. Both sides agree, confirming is consistent and integrates to as required of a density.
Topics that use this theorem
Step-by-step proofs
No step-by-step proof yet for this theorem.
References
- Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, Donald B. Rubin (2013). Bayesian Data Analysis (3rd ed.)
- Matthew D. Hoffman, Andrew Gelman (2014). The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo · arXiv:1111.4246