Kosambi–Karhunen–Loève theorem - Wikipedia

Abstraction: Optimal orthogonal decomposition of stochastic processes minimizing mean squared error

Key points:

• Represents a stochastic process as an infinite linear combination of orthogonal eigenfunctions of its covariance operator — analogous to a Fourier series but adapted to the process
• Optimality triple: KL components are (1) mutually uncorrelated, (2) concentrate maximal variance in the first K components for all K, and (3) minimize reconstruction MSE when truncated
• Empirical version is PCA; also known as Hotelling transform, proper orthogonal decomposition (POD), and empirical orthogonal functions (EOF) in geophysics
• DFT and DCT are optimal only when the covariance has circulant or centrosymmetric structure respectively; KL transform is optimal for any covariance structure
• For Gaussian processes, the KL coefficients are stochastically independent (not merely uncorrelated), enabling stronger statistical inference
• Wiener process KL expansion yields sinusoidal eigenfunctions with eigenvalues λ_k = 4/((2k-1)²π²); closely related to SVD for finite-sample data

Connections: Principal Component Analysis · Stochastic Processes · Dimensionality Reduction

Source: https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem