MathLabs
TheoremProved

Quadratic convergence of Newton's method

Statement

Suppose ff is twice continuously differentiable near a root rr with f′(r)≠0f'(r) \ne 0. Then there is a neighborhood of rr such that, if the Newton iteration starts inside it, the iterates converge to rr and satisfy ∣xn+1−r∣≤C ∣xn−r∣2|x_{n+1}-r| \le C\,|x_n-r|^2 for a constant C=max⁡∣f′′∣2min⁡∣f′∣C=\dfrac{\max|f''|}{2\min|f'|} (the maximum and minimum taken over that neighborhood).

Why is it true?

The tangent line is such a good approximation to a smooth curve that the error it makes is proportional to the square of the distance already traveled, so each correct digit roughly doubles the number of correct digits at the next step.

Proof sketch

Step 1 (Taylor expand around the current guess). Since ff is twice differentiable, Taylor's theorem with the Lagrange remainder gives f(r)=f(xn)+f′(xn)(r−xn)+12f′′(ξn)(r−xn)2f(r) = f(x_n) + f'(x_n)(r-x_n) + \tfrac12 f''(\xi_n)(r-x_n)^2 for some ξn\xi_n between xnx_n and rr. This is exact, not an approximation, because the remainder term carries the full second-order error.

Step 2 (substitute the root condition). Since f(r)=0f(r)=0, the left side vanishes, leaving 0=f(xn)+f′(xn)(r−xn)+12f′′(ξn)(r−xn)20 = f(x_n) + f'(x_n)(r-x_n) + \tfrac12 f''(\xi_n)(r-x_n)^2. Dividing through by f′(xn)f'(x_n) (nonzero near rr since f′(r)≠0f'(r) \ne 0 and f′f' is continuous) isolates r−xnr - x_n: 0=f(xn)f′(xn)+(r−xn)+f′′(ξn)2f′(xn)(r−xn)20 = \dfrac{f(x_n)}{f'(x_n)} + (r-x_n) + \dfrac{f''(\xi_n)}{2f'(x_n)}(r-x_n)^2.

Step 3 (recognize the Newton step). Rearranging, r−(xn−f(xn)f′(xn))=−f′′(ξn)2f′(xn)(r−xn)2r - \left(x_n - \dfrac{f(x_n)}{f'(x_n)}\right) = -\dfrac{f''(\xi_n)}{2f'(x_n)}(r-x_n)^2. The left-hand parenthesis is exactly the Newton update xn+1x_{n+1}, so with en=xn−re_n = x_n - r this reads r−xn+1=−f′′(ξn)2f′(xn) en2r - x_{n+1} = -\dfrac{f''(\xi_n)}{2f'(x_n)}\,e_n^2, i.e. en+1=−f′′(ξn)2f′(xn) en2e_{n+1} = -\dfrac{f''(\xi_n)}{2f'(x_n)}\,e_n^2.

Step 4 (bound the constant). Taking absolute values and bounding ∣f′′(ξn)∣|f''(\xi_n)| and 1/∣f′(xn)∣1/|f'(x_n)| by their extreme values max⁡∣f′′∣\max|f''| and 1/min⁡∣f′∣1/\min|f'| on the neighborhood gives ∣xn+1−r∣≤C ∣xn−r∣2|x_{n+1}-r| \le C\,|x_n-r|^2 with C=max⁡∣f′′∣2min⁡∣f′∣C=\dfrac{\max|f''|}{2\min|f'|}. Once C ∣x0−r∣<1C\,|x_0-r| < 1, this recursion forces ∣xn−r∣|x_n - r| to shrink to 00, proving convergence, and the squaring in ∣xn+1−r∣≤C ∣xn−r∣2|x_{n+1}-r| \le C\,|x_n-r|^2 is exactly the quadratic rate.

Topics that use this theorem

Step-by-step proofs

No step-by-step proof yet for this theorem.