MathLabs

Open problem, Analysis, posed 1949

Invariant subspace problem

Open

Let HH be a complex separable Hilbert space of dimension dim⁡H=∞\dim H = \infty and let T:H→HT : H \to H be a bounded linear operator. Does there always exist a nontrivial closed linear subspace W⊂HW \subset H (meaning W≠{0}W \neq \{0\} and W≠HW \neq H) that is invariant under TT, i.e., satisfies T(W)⊆WT(W) \subseteq W?

Research frontier as of 2026

As of 2026, the invariant subspace problem for separable infinite-dimensional complex Hilbert spaces remains open. Positive results cover wide classes of operators—including normal operators, compact and polynomially compact operators, operators commuting with compact operators, subnormal operators, and contractions whose spectrum contains the unit circle T\mathbb{T}. Preprints posted in 2023 by Per Enflo (`arXiv:2305.15442`) and by Charles W. Neville (`arXiv:2307.08176`) claiming affirmative proofs for separable Hilbert spaces have not been confirmed by peer review.

Best known results

  • Lomonosov (1973): every nonscalar bounded operator on a Banach space that commutes with a nonzero compact operator has a nontrivial closed hyperinvariant subspace.
  • Scott Brown (1978) and Brown–Chevreau–Pearcy (1988): every subnormal operator on a Hilbert space, and every contraction TT whose spectrum σ(T)\sigma(T) contains the unit circle T\mathbb{T}, has a nontrivial closed invariant subspace.
  • Enflo (1976/1987) and Read (1985): there exist bounded linear operators on separable Banach spaces (including ℓ1\ell^1) with no nontrivial closed invariant subspaces.

Tools and where they stop

ToolAchievedWhere it stops
Schauder fixed-point theorem and compact approximation (Lomonosov technique)Proves existence of hyperinvariant subspaces for operators commuting with nonzero compact operatorsMany bounded operators on HH do not commute with any nonzero compact operator
Scott Brown's dual-algebra and H∞H^\infty functional calculus methodSettles the problem for subnormal operators and contractions with rich spectrum on T\mathbb{T}Requires a rich H∞H^\infty functional calculus or spectral set structure not available for general quasinilpotent operators

Open questions

  • Does every quasinilpotent bounded linear operator TT (meaning σ(T)={0}\sigma(T) = \{0\}) on a separable infinite-dimensional Hilbert space have a nontrivial closed invariant subspace?
  • Does every bounded linear operator on the reflexive sequence space ℓp\ell^p for 1<p<∞1 < p < \infty, p≠2p \neq 2, possess a nontrivial closed invariant subspace?

References

  1. Victor I. Lomonosov (1973). Invariant subspaces for the family of operators which commute with a completely continuous operator · DOI:10.1007/BF01078890
  2. Per Enflo (1987). On the invariant subspace problem for Banach spaces · DOI:10.1007/BF02392560
  3. Heydar Radjavi, Peter Rosenthal (2003). Invariant Subspaces