Open problem, Analysis, posed 1949
Invariant subspace problem
Let be a complex separable Hilbert space of dimension and let be a bounded linear operator. Does there always exist a nontrivial closed linear subspace (meaning and ) that is invariant under , i.e., satisfies ?
As of 2026, the invariant subspace problem for separable infinite-dimensional complex Hilbert spaces remains open. Positive results cover wide classes of operators—including normal operators, compact and polynomially compact operators, operators commuting with compact operators, subnormal operators, and contractions whose spectrum contains the unit circle . Preprints posted in 2023 by Per Enflo (`arXiv:2305.15442`) and by Charles W. Neville (`arXiv:2307.08176`) claiming affirmative proofs for separable Hilbert spaces have not been confirmed by peer review.
Best known results
- Lomonosov (1973): every nonscalar bounded operator on a Banach space that commutes with a nonzero compact operator has a nontrivial closed hyperinvariant subspace.
- Scott Brown (1978) and Brown–Chevreau–Pearcy (1988): every subnormal operator on a Hilbert space, and every contraction whose spectrum contains the unit circle , has a nontrivial closed invariant subspace.
- Enflo (1976/1987) and Read (1985): there exist bounded linear operators on separable Banach spaces (including ) with no nontrivial closed invariant subspaces.
Tools and where they stop
| Tool | Achieved | Where it stops |
|---|---|---|
| Schauder fixed-point theorem and compact approximation (Lomonosov technique) | Proves existence of hyperinvariant subspaces for operators commuting with nonzero compact operators | Many bounded operators on do not commute with any nonzero compact operator |
| Scott Brown's dual-algebra and functional calculus method | Settles the problem for subnormal operators and contractions with rich spectrum on | Requires a rich functional calculus or spectral set structure not available for general quasinilpotent operators |
Open questions
- Does every quasinilpotent bounded linear operator (meaning ) on a separable infinite-dimensional Hilbert space have a nontrivial closed invariant subspace?
- Does every bounded linear operator on the reflexive sequence space for , , possess a nontrivial closed invariant subspace?
References
- Victor I. Lomonosov (1973). Invariant subspaces for the family of operators which commute with a completely continuous operator · DOI:10.1007/BF01078890
- Per Enflo (1987). On the invariant subspace problem for Banach spaces · DOI:10.1007/BF02392560
- Heydar Radjavi, Peter Rosenthal (2003). Invariant Subspaces