MathLabs
TheoremProved

Strong law of large numbers

Statement

Let X1,X2,…X_1, X_2, \dots be independent, identically distributed random variables with E[∣X1∣]<∞\mathbb{E}[|X_1|] < \infty and mean μ=E[X1]\mu = \mathbb{E}[X_1]. Then the sample mean Xˉn=1n∑i=1nXi\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i converges almost surely to μ\mu: P(lim⁡n→∞Xˉn=μ)=1\mathbb{P}\left(\lim_{n \to \infty} \bar{X}_n = \mu\right) = 1.

Why is it true?

Whereas the weak law says that at any single large step nn the average Xˉn\bar{X}_n is very likely close to μ\mu, the strong law guarantees that along almost every infinite sequence of trials, the running average eventually settles down and stays near μ\mu forever with probability 11.

Proof sketch

Without loss of generality assume μ=0\mu = 0. Truncate by setting Yn=Xn1{∣Xn∣≤n}Y_n = X_n \mathbf{1}_{\{|X_n| \le n\}}. Because ∑n=1∞P(∣Xn∣>n)=∑n=1∞P(∣X1∣>n)≤E[∣X1∣]<∞\sum_{n=1}^{\infty} \mathbb{P}(|X_n| > n) = \sum_{n=1}^{\infty} \mathbb{P}(|X_1| > n) \le \mathbb{E}[|X_1|] < \infty, the Borel–Cantelli lemma implies P(Xn≠Yn i.o.)=0\mathbb{P}(X_n \ne Y_n \text{ i.o.}) = 0. Fubini's theorem gives ∑n=1∞Var⁡(Yn)n2≤∑n=1∞E[Yn2]n2≤2E[∣X1∣]<∞\sum_{n=1}^{\infty} \frac{\operatorname{Var}(Y_n)}{n^2} \le \sum_{n=1}^{\infty} \frac{\mathbb{E}[Y_n^2]}{n^2} \le 2\mathbb{E}[|X_1|] < \infty. By Kolmogorov's convergence criterion (derived from Kolmogorov's maximal inequality), ∑n=1∞Yn−E[Yn]n\sum_{n=1}^{\infty} \frac{Y_n - \mathbb{E}[Y_n]}{n} converges almost surely. Kronecker's lemma then yields 1n∑i=1n(Yi−E[Yi])→0\frac{1}{n}\sum_{i=1}^n (Y_i - \mathbb{E}[Y_i]) \to 0 almost surely, and since E[Yn]→μ=0\mathbb{E}[Y_n] \to \mu = 0, it follows that Xˉn→0\bar{X}_n \to 0 almost surely.

Proved by

Topics that use this theorem

Related theorems

Step-by-step proofs

No step-by-step proof yet for this theorem.

References

  1. Andrey Kolmogorov (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung
  2. Rick Durrett (2019). Probability: Theory and Examples