Analysis

Partial derivatives and the gradient

Rates of change of a multivariable function along each coordinate direction, combined into the gradient vector.

IntuitionTwo slopes at once on a curved landscape

Imagine standing on a hillside whose elevation is given by a two-variable function $z = f(x,y)$ , where $x$ points east and $y$ points north. Unlike a single-variable curve, there is no single slope at your feet: stepping due east changes your height at one rate, stepping due north changes it at another, and stepping at an angle blends the two. Freezing $y$ and letting only $x$ vary slices the hillside with an east-west vertical plane and reduces the problem to an ordinary one-variable derivative.

Interactive 3D surface

z = f(x,y)

: slicing along fixed

y

reveals the curve whose slope is

\frac{\partial f}{\partial x}

, while slicing along fixed

x

gives

\frac{\partial f}{\partial y}

UndergraduateFormal definitions: partial derivatives, tangent plane, and the gradient

Definition: Partial derivative and gradient vector

Let $f : U \subseteq \mathbb{R}^n \to \mathbb{R}$ be defined on an open set $U$ . The partial derivative of $f$ with respect to $x_i$ at $a \in U$ , denoted $\frac{\partial f}{\partial x_i}(a)$ or $f_{x_i}(a)$ , is the limit keeping all other coordinates $x_j$ ( $j \ne i$ ) fixed. Assembling all $n$ partial derivatives into a single vector produces the gradient $\nabla f(a) = \left(\frac{\partial f}{\partial x_1}(a), \dots, \frac{\partial f}{\partial x_n}(a)\right)$ .

\frac{\partial f}{\partial x_i}(a) = \lim_{h \to 0} \frac{f(a_1, \dots, a_i + h, \dots, a_n) - f(a_1, \dots, a_n)}{h}

Mere existence of partial derivatives at $(x_0,y_0)$ is weaker than total differentiability. A function $f(x,y)$ is totally differentiable at $(x_0,y_0)$ if the increment $\Delta f = f(x_0+h, y_0+k) - f(x_0,y_0)$ can be written as $f_x(x_0,y_0)h + f_y(x_0,y_0)k + o\left(\sqrt{h^2+k^2}\right)$ as $(h,k) \to (0,0)$ . When $f$ is totally differentiable, the graph $z = f(x,y)$ admits a well-defined tangent plane at $(x_0,y_0)$ , and the directional derivative along any unit vector $\mathbf{u} = (u_1, u_2)$ with $\|\mathbf{u}\| = 1$ is given by the dot product $D_{\mathbf{u}}f = \nabla f \cdot \mathbf{u}$ .

z = f(x_0,y_0) + \frac{\partial f}{\partial x}(x_0,y_0)(x - x_0) + \frac{\partial f}{\partial y}(x_0,y_0)(y - y_0), \qquad D_{\mathbf{u}}f(x_0,y_0) = \nabla f(x_0,y_0) \cdot \mathbf{u}

Summary of multivariable differential operators and second-order objects
Object	Formula	Geometric / physical role
Partial derivative	$\frac{\partial f}{\partial x_i}$	Slope of the cross-section along coordinate axis $x_i$ .
Gradient vector	$\nabla f = (f_{x_1}, \dots, f_{x_n})$	Points orthogonal to level sets toward steepest ascent; norm $\\|\nabla f\\|$ is maximum rate.
Directional derivative	$D_{\mathbf{u}}f = \nabla f \cdot \mathbf{u}$	Instantaneous rate of change when moving with unit velocity $\mathbf{u}$ .
Hessian matrix & discriminant	$D = f_{xx}f_{yy} - f_{xy}^2$	Measures local curvature to classify critical points into extrema or saddle points.

UndergraduateCore theorems: Steepest ascent, Schwarz symmetry, and the Hessian test

Theorem: Directional derivative and steepest ascent along the gradient

If $f$ is differentiable at $a$ and $\nabla f(a) \ne \mathbf{0}$ , then for any unit vector $\mathbf{u}$ (with $\|\mathbf{u}\| = 1$ ), the directional derivative satisfies $D_{\mathbf{u}}f(a) = \nabla f(a) \cdot \mathbf{u} = \|\nabla f(a)\| \cos\theta$ , where $\theta$ is the angle between $\nabla f(a)$ and $\mathbf{u}$ . Consequently, $D_{\mathbf{u}}f(a)$ attains its maximum $\|\nabla f(a)\|$ when $\mathbf{u} = \frac{\nabla f(a)}{\|\nabla f(a)\|}$ , and its minimum $-\|\nabla f(a)\|$ in the opposite direction.

Why is it true?

The dot product projects the gradient onto your chosen walking direction: you gain the full magnitude of the gradient only when you align perfectly with it, and zero change when you walk perpendicular to it along a contour line.

Proof

Define the single-variable function $g(t) = f(a + t\mathbf{u})$ . By definition of the directional derivative, $D_{\mathbf{u}}f(a) = g'(0)$ . Since $f$ is totally differentiable at $a$ , we have $f(a + t\mathbf{u}) - f(a) = \nabla f(a) \cdot (t\mathbf{u}) + o(|t|)$ as $t \to 0$ . Dividing by $t$ and taking the limit $t \to 0$ yields $D_{\mathbf{u}}f(a) = \nabla f(a) \cdot \mathbf{u}$ .

By the geometric formula for the Euclidean inner product and the unit-length condition $\|\mathbf{u}\| = 1$ , we obtain $\nabla f(a) \cdot \mathbf{u} = \|\nabla f(a)\|\,\|\mathbf{u}\|\cos\theta = \|\nabla f(a)\|\cos\theta$ . Since $-1 \le \cos\theta \le 1$ , the maximum value $\|\nabla f(a)\|$ is achieved uniquely at $\theta = 0$ , where $\mathbf{u} = \frac{\nabla f(a)}{\|\nabla f(a)\|}$ , and the minimum $-\|\nabla f(a)\|$ is achieved at $\theta = \pi$ .

Theorem: Schwarz–Clairaut symmetry of mixed partial derivatives and Hessian test

If $f(x,y)$ has continuous second-order partial derivatives on a neighborhood of $(x_0,y_0)$ , then $\frac{\partial^2 f}{\partial x\partial y}(x_0,y_0) = \frac{\partial^2 f}{\partial y\partial x}(x_0,y_0)$ . Moreover, if $\nabla f(x_0,y_0) = \mathbf{0}$ and $D = f_{xx}(x_0,y_0)f_{yy}(x_0,y_0) - f_{xy}(x_0,y_0)^2$ , then $(x_0,y_0)$ is a strict local minimum if $D > 0$ and $f_{xx} > 0$ , a strict local maximum if $D > 0$ and $f_{xx} < 0$ , and a saddle point if $D < 0$ .

Why is it true?

Symmetry of mixed partials guarantees the Hessian matrix is symmetric, and completing the square on its quadratic form reveals whether the surface bowls upward, domes downward, or twists like a saddle around a flat tangent plane.

Proof

For small nonzero $h, k$ , consider the second difference $\Delta(h,k) = f(x_0+h,y_0+k) - f(x_0+h,y_0) - f(x_0,y_0+k) + f(x_0,y_0)$ . Applying the one-variable Mean Value Theorem first to $g(x) = f(x,y_0+k) - f(x,y_0)$ on $[x_0, x_0+h]$ and then to $f_x$ along $y$ yields $\Delta(h,k) = hk\,f_{yx}(c_1, d_1)$ , while applying it in the reverse order yields $\Delta(h,k) = hk\,f_{xy}(c_2, d_2)$ for intermediate points converging to $(x_0,y_0)$ . Equating the two and letting $(h,k) \to (0,0)$ proves $\frac{\partial^2 f}{\partial x\partial y}(x_0,y_0) = \frac{\partial^2 f}{\partial y\partial x}(x_0,y_0)$ by continuity.

At a critical point where $\nabla f(x_0,y_0) = \mathbf{0}$ , Taylor's formula gives $f(x_0+h,y_0+k) - f(x_0,y_0) = \frac{1}{2}Q(h,k) + o(h^2+k^2)$ , where $Q(h,k) = f_{xx}h^2 + 2f_{xy}hk + f_{yy}k^2$ . When $f_{xx} \ne 0$ , completing the square writes $Q(h,k) = \frac{1}{f_{xx}}\left[(f_{xx}h + f_{xy}k)^2 + Dk^2\right]$ with $D = f_{xx}f_{yy} - f_{xy}^2$ . If $D > 0$ , the bracketed sum is strictly positive for all $(h,k) \ne (0,0)$ , so the sign of $Q(h,k)$ matches the sign of $f_{xx}$ (giving a strict minimum for $f_{xx} > 0$ and maximum for $f_{xx} < 0$ ). If $D < 0$ , $Q(h,k)$ takes both positive and negative values along different lines through the origin, producing a saddle point.

UndergraduateReal-World Applications and Worked Examples

Partial derivatives and gradients drive modern science and engineering: in physics, Fourier's heat conduction law states that heat flux is proportional to the negative temperature gradient $-\nabla T$ ; in machine learning, training a neural network minimizes a loss function $L(\theta)$ by repeatedly stepping along $-\nabla L(\theta)$ (gradient descent); and in economics and mechanics, the Hessian test distinguishes stable equilibria (local minima of potential energy) from unstable saddle points.

Example: Directional derivative and steepest ascent of a temperature field

A metal plate has temperature $T(x,y) = x^2 + 3xy + y^3$ (in degrees Celsius) at coordinates $(x,y)$ in meters. Find the gradient $\nabla T(1,2)$ , the directional derivative at $(1,2)$ in the direction of the vector $\mathbf{v} = (3,4)$ , and the maximum rate of temperature increase at $(1,2)$ .

Solution

First, compute the partial derivatives by differentiating with respect to one variable while holding the other constant: $T_x(x,y) = 2x + 3y$ and $T_y(x,y) = 3x + 3y^2$ . Evaluating at $(1,2)$ gives $T_x(1,2) = 2(1) + 3(2) = 8$ and $T_y(1,2) = 3(1) + 3(2)^2 = 15$ , so the gradient vector is $\nabla T(1,2) = (8, 15)$ .

Next, normalize $\mathbf{v} = (3,4)$ to obtain the unit direction vector $\mathbf{u} = \frac{\mathbf{v}}{\|\mathbf{v}\|} = \left(\frac{3}{5}, \frac{4}{5}\right)$ since $\|\mathbf{v}\| = \sqrt{3^2+4^2} = 5$ . The directional derivative is $D_{\mathbf{u}}T(1,2) = \nabla T(1,2) \cdot \mathbf{u} = 8 \cdot \frac{3}{5} + 15 \cdot \frac{4}{5} = \frac{84}{5} = 16.8$ . The maximum rate of increase at $(1,2)$ occurs in the direction of $\nabla T(1,2)$ and equals $\|\nabla T(1,2)\| = \sqrt{8^2 + 15^2} = \sqrt{289} = 17$ degrees Celsius per meter.

Example: Classifying critical points with the Hessian determinant

Find and classify all critical points of the function $f(x,y) = x^3 + y^3 - 3xy$ using the second-derivative Hessian test.

Solution

Set both first partial derivatives to zero: $f_x(x,y) = 3x^2 - 3y = 0$ and $f_y(x,y) = 3y^2 - 3x = 0$ . From the first equation we have $y = x^2$ ; substituting into the second gives $3(x^2)^2 - 3x = 3x(x^3 - 1) = 0$ , which has real solutions $x = 0$ and $x = 1$ . Thus the two critical points are $(0,0)$ and $(1,1)$ .

Now compute the second partials: $f_{xx} = 6x$ , $f_{yy} = 6y$ , and $f_{xy} = -3$ , giving the Hessian discriminant $D(x,y) = f_{xx}f_{yy} - f_{xy}^2 = 36xy - 9$ . At $(0,0)$ , we have $D(0,0) = -9 < 0$ , so $(0,0)$ is a saddle point. At $(1,1)$ , we have $D(1,1) = 36 - 9 = 27 > 0$ and $f_{xx}(1,1) = 6 > 0$ , so $(1,1)$ is a strict local minimum with minimum value $f(1,1) = -1$ .

Let $f(x,y) = x^3 y^2 - 4x + 5y$ . What is the gradient vector $\nabla f(1, 2)$ ?

If $\nabla f(x_0, y_0) = (6, -8)$ , what is the maximum possible directional derivative $D_{\mathbf{u}}f(x_0, y_0)$ over all unit vectors $\mathbf{u}$ ?

In machine learning, gradient descent updates parameters $\theta$ to minimize a differentiable loss function $L(\theta)$ . Which update step moves $\theta$ in the direction of steepest local decrease of $L$ ?

Suppose $(x_0, y_0)$ is a critical point of a smooth function $f(x,y)$ where $f_{xx}(x_0,y_0) = -4$ , $f_{yy}(x_0,y_0) = -3$ , and $f_{xy}(x_0,y_0) = 2$ . How is $(x_0, y_0)$ classified?

References

Jerrold E. Marsden, Anthony J. Tromba (2012). Vector Calculus
James Stewart (2015). Calculus: Early Transcendentals
Augustin-Louis Cauchy (1847). Méthode générale pour la résolution des systèmes d'équations simultanées