MathLabs
TheoremProved

Weak law of large numbers

Statement

Let X1,X2,…X_1, X_2, \dots be independent, identically distributed random variables with finite mean μ=E[X1]\mu = \mathbb{E}[X_1]. For the sample mean Xˉn=1n∑i=1nXi\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i and every ε>0\varepsilon > 0, we have lim⁡n→∞P(∣Xˉn−μ∣≥ε)=0\lim_{n \to \infty} \mathbb{P}(|\bar{X}_n - \mu| \ge \varepsilon) = 0; that is, Xˉn\bar{X}_n converges in probability to μ\mu.

Why is it true?

Any single trial of a random experiment can fluctuate wildly, but when you average many independent trials together, positive and negative deviations tend to cancel out, making a large departure of the sample average from the true mean μ\mu increasingly unlikely as the sample size nn grows.

Proof sketch

When the variance σ2=Var⁡(X1)\sigma^2 = \operatorname{Var}(X_1) is finite, linearity and independence give E[Xˉn]=μ\mathbb{E}[\bar{X}_n] = \mu and Var⁡(Xˉn)=σ2n\operatorname{Var}(\bar{X}_n) = \frac{\sigma^2}{n}. Applying Chebyshev's inequality yields P(∣Xˉn−μ∣≥ε)≤Var⁡(Xˉn)ε2=σ2nε2→0\mathbb{P}(|\bar{X}_n - \mu| \ge \varepsilon) \le \frac{\operatorname{Var}(\bar{X}_n)}{\varepsilon^2} = \frac{\sigma^2}{n\varepsilon^2} \to 0 as n→∞n \to \infty. When only E[∣X1∣]<∞\mathbb{E}[|X_1|] < \infty is assumed (Khinchin's theorem), one truncates at level nn by setting Yi=Xi1{∣Xi∣≤n}Y_{i} = X_i \mathbf{1}_{\{|X_i| \le n\}}, uses E[∣X1∣]<∞\mathbb{E}[|X_1|] < \infty to show P(∃i≤n:Xi≠Yi)≤nP(∣X1∣>n)→0\mathbb{P}(\exists i \le n : X_i \ne Y_i) \le n\mathbb{P}(|X_1| > n) \to 0, and applies Chebyshev's inequality to the truncated average.

Topics that use this theorem

Related theorems

Step-by-step proofs

No step-by-step proof yet for this theorem.

References

  1. Jacob Bernoulli (1713). Ars Conjectandi
  2. Geoffrey Grimmett, David Stirzaker (2020). Probability and Random Processes