Among all lines y=b0+b1x, the choice that minimizes the sum of squared residuals ∑i=1n(yi−b0−b1xi)2 is b1=β^1=SxxSxy and b0=β^0=yˉ−β^1xˉ, where Sxy=∑i=1n(xi−xˉ)(yi−yˉ) and Sxx=∑i=1n(xi−xˉ)2.
Why is it true?
The sum of squared residuals is a smooth (quadratic, convex) function of b0 and b1, so its minimum is found exactly where both partial derivatives vanish — the same idea as finding the bottom of a bowl by setting its slope to zero in every direction.
Proof sketch
Write Q(b0,b1)=∑i=1n(yi−b0−b1xi)2. Taking the partial derivative with respect to b0 and setting it to zero: ∂b0∂Q=−2∑i=1n(yi−b0−b1xi)=0, which simplifies to the first normal equation ∑i=1nyi=nb0+b1∑i=1nxi, i.e. b0=yˉ−b1xˉ.
Taking the partial derivative with respect to b1 and setting it to zero: ∂b1∂Q=−2∑i=1nxi(yi−b0−b1xi)=0, the second normal equation ∑i=1nxiyi=b0∑i=1nxi+b1∑i=1nxi2.
Substitute b0=yˉ−b1xˉ from the first equation into the second: ∑i=1nxiyi=(yˉ−b1xˉ)nxˉ+b1∑i=1nxi2=nxˉyˉ+b1(∑i=1nxi2−nxˉ2). Rearranging isolates b1: b1(∑i=1nxi2−nxˉ2)=∑i=1nxiyi−nxˉyˉ.
A direct algebraic expansion shows ∑i=1nxi2−nxˉ2=∑i=1n(xi−xˉ)2=Sxx and ∑i=1nxiyi−nxˉyˉ=∑i=1n(xi−xˉ)(yi−yˉ)=Sxy, so b1=Sxy/Sxx. Substituting back gives b0=yˉ−b1xˉ. Because Q is a sum of squares in a quadratic form that grows without bound as ∣b0∣,∣b1∣→∞, this unique stationary point must be the global minimum.