TheoremProved

Bayes' theorem

Statement

For any events $A$ and $B$ with $\mathbb{P}(A) > 0$ and $\mathbb{P}(B) > 0$ , $\mathbb{P}(A \mid B) = \frac{\mathbb{P}(B \mid A)\,\mathbb{P}(A)}{\mathbb{P}(B)}$ . More generally, if $A_1, \dots, A_k$ form a partition of the sample space with $\mathbb{P}(A_i) > 0$ and $\mathbb{P}(B) > 0$ , then for each $j$ , $\mathbb{P}(A_j \mid B) = \frac{\mathbb{P}(B \mid A_j)\,\mathbb{P}(A_j)}{\sum_{i=1}^k \mathbb{P}(B \mid A_i)\,\mathbb{P}(A_i)}$ .

Why is it true?

Bayes' theorem is the mathematical rule for reversing conditional probabilities: it tells you how to update a prior belief $\mathbb{P}(A)$ about a hidden cause $A$ into a posterior belief $\mathbb{P}(A \mid B)$ after observing evidence $B$ , by weighting the prior by how likely $A$ was to produce $B$ .

Proof sketch

By the definition of conditional probability, $\mathbb{P}(A \mid B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}$ when $\mathbb{P}(B) > 0$ , and symmetrically $\mathbb{P}(B \mid A) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)}$ when $\mathbb{P}(A) > 0$ . Equating the two expressions for $\mathbb{P}(A \cap B)$ gives $\mathbb{P}(A \cap B) = \mathbb{P}(B \mid A)\,\mathbb{P}(A)$ , and dividing by $\mathbb{P}(B)$ yields $\mathbb{P}(A \mid B) = \frac{\mathbb{P}(B \mid A)\,\mathbb{P}(A)}{\mathbb{P}(B)}$ . If $A_1, \dots, A_k$ partition the sample space, the law of total probability expands the denominator as $\mathbb{P}(B) = \sum_{i=1}^k \mathbb{P}(B \cap A_i) = \sum_{i=1}^k \mathbb{P}(B \mid A_i)\,\mathbb{P}(A_i)$ .

Proved by

proved 1763Pierre-Simon Laplace

Topics that use this theorem

Conditional probability and Bayes' theorem

Related theorems

Kolmogorov's axioms of probability

Step-by-step proofs

No step-by-step proof yet for this theorem.

References

Thomas Bayes, Richard Price (1763). An Essay towards solving a Problem in the Doctrine of Chances · DOI:10.1098/rstl.1763.0053
Pierre-Simon Laplace (1812). Théorie analytique des probabilités