Analysis

Functional analysis

Studies infinite-dimensional vector spaces of functions and the linear operators acting on them.

IntuitionWhen can we do calculus with infinitely many coordinates?

A vector in $\mathbb{R}^n$ is just a list of $n$ numbers, and calculus on $\mathbb{R}^n$ works because we can measure length ( $\|x\|$ ) and angle (inner product) between such lists. A function like a sound wave, an image, or a quantum wavefunction is really an infinite list of numbers — its value at every point, or equivalently its infinitely many Fourier coefficients. Functional analysis asks: can we still measure "length" and "angle" between functions, and still do calculus (limits, derivatives, optimization) on the resulting infinite-dimensional space? The answer is yes, provided the space is complete — infinite sums and limits do not "leak out" of the space — which is exactly what turns a plain vector space of functions into a Banach space or, when an inner product is available, a Hilbert space.

The network of inclusions between spaces of functions and sequences: every Hilbert space (node

\ell^2

, node

L^2[0,1]

) is a Banach space, and every Banach space (also

C[0,1]

and

\ell^p

) is a normed vector space, but the arrows do not reverse —

C[0,1]

with the sup norm has no inner product that induces it. Highlighting a node traces which broader class it belongs to.

UndergraduateNorms, completeness, and inner products

Definition: Normed vector space and Banach space

A norm on a vector space $X$ over $\mathbb{R}$ or $\mathbb{C}$ is a map $\|x\|$ satisfying $\|x\|$ $=0$ iff $x=0$ , $\|\lambda x\|=|\lambda|\|x\|$ , and the triangle inequality $\|x+y\|\le\|x\|+\|y\|$ . A sequence $(x_n)$ is a Cauchy sequence if its terms eventually get arbitrarily close together; $X$ is complete, and called a Banach space, if every Cauchy sequence in $X$ converges to some point of $X$ (not just of a larger space containing it). Completeness is what lets us build a limit function out of an infinite process — an infinite series of functions, an iterative approximation scheme — and be sure the limit is still a bona fide element of the space.

Written out, the Cauchy condition says the tail of the sequence gets tightly bunched together in norm:

\forall\varepsilon>0\ \exists N:\ n,m\ge N \Longrightarrow \|x_n-x_m\|<\varepsilon

Here $\varepsilon$ is the tolerance, $N$ is the point beyond which every pair of terms is within that tolerance, and $\|x\|$ is the norm on $X$ . When $X$ additionally carries an inner product $\langle x,x\rangle$ — a bilinear (or sesquilinear) pairing generalizing the dot product, from which the norm is recovered as $\|x\|=\sqrt{\langle x,x\rangle}$ — and is complete in that induced norm, $X$ is called a Hilbert space, usually written $H$ . The inner product is what lets us speak of orthogonality and angle between functions, not just their length.

|\langle x,y\rangle|\le\|x\|\,\|y\|

This is the Cauchy–Schwarz inequality: the inner product of two vectors can never exceed the product of their lengths, exactly like $\vec u\cdot\vec v=\|\vec u\|\|\vec v\|\cos\theta$ in ordinary 3D space, where $|\cos\theta|\le1$ . Hilbert spaces are further characterized among Banach spaces by the parallelogram law, $\|x+y\|^2+\|x-y\|^2=2\|x\|^2+2\|y\|^2$ — a purely norm-based identity (no inner product needed to state it) that a norm must satisfy if and only if it actually comes from an inner product. It says the sum of the squared diagonals of a parallelogram equals the sum of the squared sides, generalizing the Pythagorean-flavored geometry of Euclidean space to infinite dimensions.

Common Banach and Hilbert spaces
Space	Norm	Complete?	Hilbert space?
$\ell^2$	$\left(\sum_i\|x_i\|^2\right)^{1/2}$	Yes	Yes
$\ell^p$ , $p\ne2$	$\left(\sum_i\|x_i\|^p\right)^{1/p}$	Yes	No
$C[0,1]$	$\sup_{t}\|f(t)\|$	Yes	No
$L^2[0,1]$	$\left(\int_0^1\|f(t)\|^2dt\right)^{1/2}$	Yes	Yes

UndergraduateTwo pillars: the Riesz representation theorem and the Hahn–Banach theorem

Theorem: Riesz representation theorem (Hilbert space case)

Let $H$ be a Hilbert space and $\varphi$ a bounded (continuous) linear functional on $H$ . Then there is a unique $y\in H$ such that $\varphi(x)=\langle x,y\rangle$ for every $x\in H$ , and moreover $\|\varphi\|=\|y\|$ .

Why is it true?

It says every way of assigning a number to each vector linearly and continuously is secretly just "taking the inner product with some fixed vector" — abstract functionals are no more general than the most concrete ones you already know.

Proof

If $\varphi=0$ , take $y=0$ and we are done. Otherwise let $N=\ker\varphi=\{x\in H:\varphi(x)=0\}$ ; since $\varphi$ is continuous and linear, $N$ is a closed proper subspace of $H$ . Because $H$ is a Hilbert space, the projection theorem gives an orthogonal decomposition $H=N\oplus N^{\perp}$ , and since $N$ is a proper subspace, $N^{\perp}$ contains some $z\ne0$ .

For any $x\in H$ , consider the vector $u=\varphi(x)z-\varphi(z)x$ . Applying $\varphi$ gives $\varphi(u)=\varphi(x)\varphi(z)-\varphi(z)\varphi(x)=0$ , so $u\in N$ . Since $z\in N^{\perp}$ , we get $\langle u,z\rangle=0$ , i.e. $\varphi(x)\langle z,z\rangle-\varphi(z)\langle x,z\rangle=0$ . Solving for $\varphi(x)$ gives $\varphi(x)=\dfrac{\varphi(z)}{\|z\|^2}\langle x,z\rangle=\left\langle x,\dfrac{\overline{\varphi(z)}}{\|z\|^2}z\right\rangle$ , so $y=\dfrac{\overline{\varphi(z)}}{\|z\|^2}z$ represents $\varphi$ .

For uniqueness, if $\langle x,y_1\rangle=\langle x,y_2\rangle$ for all $x$ , take $x=y_1-y_2$ to get $\|y_1-y_2\|^2=0$ , so $y_1=y_2$ . For the norm identity, Cauchy–Schwarz gives $|\varphi(x)|=|\langle x,y\rangle|\le\|y\|\|x\|$ , so $\|\varphi\|\le\|y\|$ ; and plugging in $x=y$ gives $\varphi(y)=\|y\|^2$ , so $\|\varphi\|\ge|\varphi(y)|/\|y\|=\|y\|$ . Together, $\|\varphi\|=\|y\|$ .

Theorem: Hahn–Banach extension theorem

Let $X$ be a real normed vector space, $Y$ a linear subspace of $X$ , and $\varphi$ a bounded linear functional on $Y$ with $|\varphi(y)|\le M\|y\|$ for all $y\in Y$ . Then there exists a bounded linear functional $\Phi$ on all of $X$ such that $\Phi(y)=\varphi(y)$ for every $y\in Y$ , and $|\Phi(x)|\le M\|x\|$ for every $x\in X$ .

Why is it true?

It guarantees that a linear functional defined only on a small subspace — for instance, only knowing "the value of a signal at a few sample points" — can always be extended to the whole space without growing its bound; nothing ever forces us to be stuck working on a subspace.

Proof

First extend by one dimension. Pick $x_0\notin Y$ and let $Y_1=Y\oplus\mathbb{R}x_0$ . We must choose the value $c=\Phi(x_0)$ so that $|\varphi(y)+tc|\le M\|y+tx_0\|$ for all $y\in Y,t\in\mathbb{R}$ ; dividing by $t\ne0$ and substituting $y/t\to y$ , this reduces to needing $c$ with $\varphi(y)-M\|y-x_0\|\le c\le M\|y+x_0\|-\varphi(y)$ for all $y\in Y$ . Using $\varphi(y_1)-\varphi(y_2)=\varphi(y_1-y_2)\le M\|y_1-y_2\|\le M\|y_1+x_0\|+M\|y_2-x_0\|$ , one checks the supremum of the left expressions over $y_1$ never exceeds the infimum of the right expressions over $y_2$ , so a valid $c$ in that interval exists; this defines $\Phi$ on $Y_1$ with the same bound $M$ .

Next, extend to all of $X$ using Zorn's Lemma. Consider the set of all pairs $(Z,\Psi)$ where $Z$ is a subspace with $Y\subseteq Z\subseteq X$ and $\Psi$ extends $\varphi$ to $Z$ with bound $M$ , partially ordered by $(Z_1,\Psi_1)\le(Z_2,\Psi_2)$ iff $Z_1\subseteq Z_2$ and $\Psi_2|_{Z_1}=\Psi_1$ . Every chain has an upper bound (take the union of the subspaces and the functional agreeing with each on its domain), so Zorn's Lemma gives a maximal element $(Z^*,\Phi)$ .

Finally, if $Z^*\ne X$ , the one-dimension extension step above applied to $Z^*$ and any $x_0\in X\setminus Z^*$ would produce a strictly larger admissible pair, contradicting maximality of $(Z^*,\Phi)$ . Hence $Z^*=X$ , and $\Phi$ is the desired bound-preserving extension to all of $X$ .

UndergraduateReal-World Applications and Worked Examples

Functional analysis is the mathematical backbone of Fourier analysis and signal processing (a signal lives in $L^2[0,2\pi]$ or $L^2[0,1]$ ), of quantum mechanics (states live in a Hilbert space $H$ ), and of statistics, engineering, and machine learning problems that boil down to finding the closest point in a subspace — from least-squares regression to Tikhonov-regularized inverse problems.

Example: Signal processing: does $\sin(nt)$ settle down as $n\to\infty$ ?

Fix $f\in L^2[0,2\pi]$ . Show that $\int_0^{2\pi} f(t)\sin(nt)\,dt\to 0$ as $n\to\infty$ (i.e. $\sin(nt)\rightharpoonup 0$ weakly), while the norm $\|\sin(nt)\|_2=\sqrt{\pi}$ stays constant — so $\sin(nt)$ itself never settles down in the strong (norm) sense.

Solution

Consider the orthonormal system $e_n(t)=\sin(nt)/\sqrt\pi$ in $L^2[0,2\pi]$ (orthogonality follows from $\int_0^{2\pi}\sin(nt)\sin(mt)\,dt=0$ for $n\ne m$ , and $\int_0^{2\pi}\sin^2(nt)\,dt=\pi$ ).

By Bessel's inequality, $\sum_{n=1}^{\infty}|\langle f,e_n\rangle|^2\le\|f\|^2$ , the series of squared Fourier coefficients of $f$ against this orthonormal system converges, since $\|f\|^2<\infty$ . A convergent series has terms tending to zero, so $|\langle f,e_n\rangle|^2\to0$ , i.e. $\langle f,e_n\rangle\to0$ .

Since $\langle f,e_n\rangle=\frac{1}{\sqrt\pi}\int_0^{2\pi}f(t)\sin(nt)\,dt$ , this is exactly $\int_0^{2\pi} f(t)\sin(nt)\,dt\to 0$ — this is the Riemann–Lebesgue lemma in disguise, and precisely the statement $\sin(nt)\rightharpoonup 0$ .

On the other hand, direct computation gives $\|\sin(nt)\|_2^2=\int_0^{2\pi}\sin^2(nt)\,dt=\pi$ for every $n$ , so $\|\sin(nt)\|_2=\sqrt{\pi}$ for all $n$ — the norm never shrinks. So $\sin(nt)$ converges to $0$ when tested against any fixed $f$ (weak convergence), but never converges to $0$ in norm (no strong convergence): the oscillations get faster and "average out" against every fixed probe, without the signal itself losing energy.

Example: Statistics and engineering: least-squares regression as an orthogonal projection

Given data matrix $A$ (columns = predictors) and observations $b$ , the least-squares fit minimizes $\|Ax-b\|^2$ over $x$ . Use the Hilbert-space projection idea (the finite-dimensional case of the Riesz/orthogonality argument above, with $H=\mathbb{R}^n$ and the dot product) to derive the normal equations $A^{\mathsf T}A\hat x=A^{\mathsf T}b$ that the minimizer $\hat x$ must satisfy.

Solution

The set of achievable outputs $\{Ax:x\in\mathbb{R}^n\}$ is a subspace $\mathrm{ran}(A)\subseteq\mathbb{R}^m$ (a finite-dimensional, automatically complete, hence closed subspace of the Hilbert space $\mathbb{R}^m$ ). Minimizing $\|Ax-b\|^2$ is exactly the problem of finding the point of $\mathrm{ran}(A)$ closest to $b$ .

By the Hilbert-space projection theorem (the same orthogonal-decomposition idea used to prove the Riesz representation theorem above), the closest point $A\hat x$ in a closed subspace to $b$ is characterized by the residual $b-A\hat x$ being orthogonal to the entire subspace, i.e. $\langle b-A\hat x,\,Av\rangle=0\ \ \forall v$ .

Writing $Av=A(v_1,\dots,v_n)$ ranges over all columns of $A$ as $v$ ranges over the standard basis vectors, the condition $\langle b-A\hat x,\,Av\rangle=0\ \ \forall v$ applied to each column $a_j$ of $A$ says $\langle b-A\hat x,a_j\rangle=0$ for every $j$ , which packaged as a single matrix equation is exactly $A^{\mathsf T}(b-A\hat x)=0$ .

Expanding gives $A^{\mathsf T}b-A^{\mathsf T}A\hat x=0$ , i.e. the normal equations $A^{\mathsf T}A\hat x=A^{\mathsf T}b$ . So the abstract Hilbert-space fact "the residual of a best approximation is orthogonal to the approximating subspace" is, in disguise, exactly the linear-algebra formula used every day to fit a regression line or a financial factor model.

Common mistake. Do not confuse weak convergence (

\sin(nt)\rightharpoonup 0

, meaning

\langle x_n,y\rangle\to\langle x,y\rangle

against every fixed

y

) with strong (norm) convergence (

\|x_n-x\|\to0

). The example above shows the gap concretely:

\sin(nt)\rightharpoonup0

weakly in

L^2[0,2\pi]

, yet

\|\sin(nt)\|_2=\sqrt\pi\ne0

for every

n

, so it does not converge to

0

in norm at all. Strong convergence always implies weak convergence (by Cauchy–Schwarz,

|\langle x_n-x,y\rangle|\le\|x_n-x\|\|y\|\to0

), but the converse fails exactly when "mass" escapes to infinite frequency, infinite index, or spreads out thinly — which is precisely what compactness (needed for the spectral theorem in the next topic) is designed to rule out.

Historical note

David Hilbert's work on integral equations in the early 1900s — studying operators like $Tf(x)=\int K(x,y)f(y)\,dy$ — led him to treat sequences of Fourier-like coefficients as points of an infinite-dimensional space, the object later named a Hilbert space in his honor. In parallel, Maurice Fréchet and Frigyes Riesz developed the abstract theory of normed and metric function spaces (Riesz's own representation result for $L^2[0,1]$ dates to 1907), and Stefan Banach's 1932 book Théorie des opérations linéaires crystallized the general axioms of what we now call a Banach space. It was John von Neumann, in his 1929–1932 work on the mathematical foundations of quantum mechanics, who first gave a fully abstract, axiomatic definition of a Hilbert space independent of any specific realization — precisely because he needed a rigorous setting in which Heisenberg's matrix mechanics and Schrödinger's wave mechanics could be shown to be the same physical theory.

David Hilbert John von Neumann

ResearchOpen directions: the geometry of infinite-dimensional spaces

Research frontier as of 2026

A major active theme is the Ribe program: Martin Ribe showed in 1976 that the local (finite-dimensional-subspace) structure of a Banach space is, remarkably, determined entirely by its structure as a metric space, with no reference to linearity. Since then, researchers (notably Assaf Naor and collaborators) have worked to translate classical Banach-space invariants — type, cotype, smoothness — into purely metric notions (metric type, metric cotype, Markov type) that make sense for any metric space, not just linear ones. This nonlinear geometric functional analysis now feeds directly into theoretical computer science: bounds on how well a metric space can be embedded into $\ell^2$ or $\ell^1$ with low distortion control the performance of approximation algorithms for problems like Sparsest Cut, and quantitative nonembeddability results (e.g. for expander graphs) give lower bounds for such algorithms. As of 2026, precise quantitative versions of several nonlinear Dvoretzky-type theorems, and the full resolution of Bourgain's discretization problem in general Banach spaces, remain open lines of research at the interface of functional analysis, metric geometry, and computer science.

Which of the following Banach spaces is not a Hilbert space (its norm does not come from an inner product)?

In a Hilbert space, if $\|x\|=3$ and $\|y\|=4$ , what is $\|x+y\|^2+\|x-y\|^2$ ?

What does the Riesz representation theorem guarantee about a bounded linear functional $\varphi$ on a Hilbert space $H$ ?

In the $\sin(nt)$ example, why does the signal $\sin(nt)$ "average out to zero" against every fixed probe $f$ as $n\to\infty$ , even though it never loses energy?

References

Walter Rudin (1991). Functional Analysis
John B. Conway (2007). A Course in Functional Analysis
Assaf Naor (2012). An introduction to the Ribe program · arXiv:1205.5993