Fermat's interior extremum theorem and the second-derivative test

(Necessity.) Suppose $x_0$ is a local maximum; the local-minimum case is symmetric. There is $\delta>0$ with $f(x)\le f(x_0)$ for all $x$ with $|x-x_0|<\delta$. For $h\in(0,\delta)$, $\dfrac{f(x_0+h)-f(x_0)}{h}\le 0$, and letting $h\to 0^+$ gives $f'(x_0)\le 0$. For $h\in(-\delta,0)$, the numerator is $\le 0$ and the denominator is negative, so $\dfrac{f(x_0+h)-f(x_0)}{h}\ge 0$, and letting $h\to 0^-$ gives $f'(x_0)\ge 0$. Since $f$ is differentiable at $x_0$, both one-sided limits agree, forcing $f'(x_0)=0$.

(Sufficiency, case $f''(x_0)>0$.) Since $f'(x_0)=0$ and $f''$ exists near $x_0$, the definition of $f''$ as the derivative of $f'$ gives $f'(x)=f'(x_0)+f''(x_0)(x-x_0)+o(x-x_0)=f''(x_0)(x-x_0)+o(x-x_0)$ as $x\to x_0$. Because $f''(x_0)>0$, this expression is negative for $x$ slightly less than $x_0$ and positive for $x$ slightly greater than $x_0$.

So $f'$ changes sign from $-$ to $+$ across $x_0$: by the monotonicity theorem, $f$ is decreasing just to the left of $x_0$ and increasing just to the right, which is exactly the definition of a local minimum at $x_0$.

The case $f''(x_0)<0$ is identical with all inequalities reversed, giving $f'$ changing from $+$ to $-$, hence a local maximum. When $f''(x_0)=0$ the expansion above gives no information about the sign of $f'$ near $x_0$, so the test is inconclusive and one must examine the sign of $f'$ directly (as the pitfall below illustrates with $f(x)=x^3$).