TheoremProved

Central limit theorem

Statement

Let $X_1, X_2, \dots$ be independent, identically distributed random variables with finite mean $\mu = \mathbb{E}[X_1]$ and finite positive variance $\sigma^2 = \operatorname{Var}(X_1) > 0$ . For the standardized sum $Z_n = \frac{\sum_{i=1}^n X_i - n\mu}{\sigma\sqrt{n}} = \frac{\sqrt{n}(\bar{X}_n - \mu)}{\sigma}$ , we have $\lim_{n \to \infty} \mathbb{P}(Z_n \le z) = \Phi(z) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^z e^{-t^2/2}\,dt$ for every $z \in \mathbb{R}$ ; that is, $Z_n$ converges in distribution to $\mathcal{N}(0,1)$ .

Why is it true?

When you add up many small, independent random effects, the quirks of their individual distributions wash out, and the rescaled fluctuation around the mean universally settles into the bell-shaped Gaussian curve $\mathcal{N}(0,1)$ — which is why measurement errors, test scores, and thermal noise all look approximately normal.

Proof sketch

Set $Y_i = \frac{X_i - \mu}{\sigma}$ , so $\mathbb{E}[Y_i] = 0$ , $\operatorname{Var}(Y_i) = 1$ , and $Z_n = \frac{1}{\sqrt{n}}\sum_{i=1}^n Y_i$ . Let $\varphi_Y(t) = \mathbb{E}[e^{itY_1}]$ be the characteristic function of $Y_1$ . Since $\mathbb{E}[Y_1^2] = 1 < \infty$ , a second-order Taylor expansion at $t = 0$ gives $\varphi_Y(t) = 1 - \frac{t^2}{2} + o(t^2)$ as $t \to 0$ . By independence, the characteristic function of $Z_n$ is $\varphi_{Z_n}(t) = \left[\varphi_Y\left(\frac{t}{\sqrt{n}}\right)\right]^n = \left(1 - \frac{t^2}{2n} + o\left(\frac{1}{n}\right)\right)^n \to e^{-t^2/2}$ as $n \to \infty$ for each fixed $t \in \mathbb{R}$ . Because $e^{-t^2/2}$ is the characteristic function of $\mathcal{N}(0,1)$ , Lévy's continuity theorem implies $Z_n \xrightarrow{d} \mathcal{N}(0,1)$ .

Topics that use this theorem

The law of large numbers and the central limit theorem

Related theorems

Weak law of large numbers Strong law of large numbers Wigner's semicircle law

Step-by-step proofs

No step-by-step proof yet for this theorem.

References

Abraham de Moivre (1738). The Doctrine of Chances · DOI:10.1016/b978-044450871-3/50088-7
Pierre-Simon Laplace (1812). Théorie analytique des probabilités
Patrick Billingsley (1995). Probability and Measure