Eckart–Young–Mirsky theorem

Achievability. Since $U, V$ have orthonormal columns, $\|A - A_k\|_2 = \|U(\Sigma - \Sigma_k)V^\top\|_2 = \|\Sigma - \Sigma_k\|_2$, where $\Sigma_k$ keeps the top $k$ singular values and zeroes the rest. $\Sigma - \Sigma_k$ is diagonal with entries $0, \dots, 0, \sigma_{k+1}, \dots, \sigma_r$, so its operator norm is its largest entry, $\sigma_{k+1}$.

Optimality. Let $B$ be any matrix with $\operatorname{rank}(B) \le k$; its null space (kernel) has dimension at least $n - k$. Let $S = \operatorname{span}(v_1, \dots, v_{k+1})$, a $(k+1)$-dimensional subspace. Since $\dim(\ker B) + \dim(S) \ge (n-k) + (k+1) = n+1 > n$, the two subspaces must intersect in more than just the origin: there exists a unit vector $z \in S$ with $Bz = 0$.

Write $z = \sum_{i=1}^{k+1} c_i v_i$ with $\sum_{i=1}^{k+1} c_i^2 = 1$ (since $z$ has unit norm and the $v_i$ are orthonormal). Because $Bz = 0$, $\|(A-B)z\| = \|Az\| = \left\|\sum_{i=1}^{k+1} c_i \sigma_i u_i\right\| = \sqrt{\sum_{i=1}^{k+1} c_i^2 \sigma_i^2}$, using orthonormality of the $u_i$.

Since $\sigma_i \ge \sigma_{k+1}$ for every $i \le k+1$, $\sum_{i=1}^{k+1} c_i^2 \sigma_i^2 \ge \sigma_{k+1}^2 \sum_{i=1}^{k+1} c_i^2 = \sigma_{k+1}^2$. Hence $\|A-B\|_2 \ge \|(A-B)z\| \ge \sigma_{k+1}$ for every rank-$\le k$ matrix $B$, which together with achievability proves the theorem.