TheoremProved

Bayes' theorem

Statement

Let $B_1, B_2, \ldots, B_n$ be a partition of $\Omega$ with $P(B_i) > 0$ for every $i$ , and let $A$ be an event with $P(A) > 0$ . Then for each $i$ , $P(B_i \mid A) = \frac{P(B_i) \cdot P(A \mid B_i)}{\sum_{j=1}^{n} P(B_j) \cdot P(A \mid B_j)}$ .

Why is it true?

Bayes' theorem reverses the direction of conditioning: it starts from how likely the evidence $A$ is under each hypothesis $B_i$ (often easy to know or measure) and returns how likely each hypothesis is once the evidence $A$ has actually been observed (usually what we really want to know), by re-weighting each prior probability $P(B_i)$ according to how well it explains $A$ .

Proof sketch

Step 1 (write $P(B_i \cap A)$ two ways): by the multiplication rule applied in both directions, $P(B_i \cap A) = P(B_i) \cdot P(A \mid B_i)$ and also $P(B_i \cap A) = P(A) \cdot P(B_i \mid A)$ , since $P(A) > 0$ .

Step 2 (equate the two expressions and solve): setting the two expressions for $P(B_i \cap A)$ equal gives $P(A) \cdot P(B_i \mid A) = P(B_i) \cdot P(A \mid B_i)$ , and dividing both sides by $P(A)$ yields $P(B_i \mid A) = \frac{P(B_i) \cdot P(A \mid B_i)}{P(A)}$ .

Step 3 (expand the denominator with the law of total probability): since $B_1, \ldots, B_n$ partition $\Omega$ , the law of total probability gives $P(A) = \sum_{j=1}^{n} P(B_j) \cdot P(A \mid B_j)$ .

Step 4 (substitute to finish): replacing $P(A)$ in Step 2 by this sum gives exactly $P(B_i \mid A) = \frac{P(B_i) \cdot P(A \mid B_i)}{\sum_{j=1}^{n} P(B_j) \cdot P(A \mid B_j)}$ , which is Bayes' theorem.

Topics that use this theorem

Conditional probability and Bayes' theorem

Step-by-step proofs

No step-by-step proof yet for this theorem.