Principle of transfinite induction

Suppose, for contradiction, that $C$ does not contain every ordinal. Then the class $D$ of ordinals not in $C$ is nonempty. The ordinals themselves form a well-order (any nonempty class of ordinals has a least element — this is essentially the defining property of the ordinals), so $D$ has a least element; call it $\alpha$.

By minimality of $\alpha$, every ordinal $\beta<\alpha$ is not in $D$, i.e. every $\beta<\alpha$ satisfies $\beta \in C$.

But this is exactly the hypothesis of the theorem applied to $\alpha$: "$\beta \in C$ for all $\beta<\alpha$" implies $\alpha \in C$. So $\alpha \in C$.

This contradicts $\alpha \in D$ (i.e. $\alpha \notin C$). The contradiction shows no such least counterexample $\alpha$ can exist, so $D=\emptyset$: $C$ contains every ordinal. $\blacksquare$