#009 The Singular Value Decomposition(SVD) - illustrated in Python - Master Data Science
svdlinear-algebrapythonimage-compressionnumpy
Abstraction: Step-by-step SVD tutorial with Python implementation and image reconstruction examples
Key points:
- SVD decomposes matrix A into U Σ V^T; U and V are unitary (orthonormal), Σ is diagonal with singular values in decreasing order
- Left singular vectors (U) are eigenvectors of AA^T; right singular vectors (V) are eigenvectors of A^T A; singular values σ are square roots of the shared eigenvalues
- Rank-r approximation keeps only r largest singular value terms, enabling lossy compression; singular values typically decay rapidly
- Applied to the Olivetti faces dataset (400 images, 64×64) showing progressive reconstruction quality as r increases
- Eigenfaces (columns of U) form a basis for face images — central to face recognition methods
- Python implementation uses
np.linalg.svd; truncation by zeroing small sigma values avoids numerical noise
Connections: Numpy · Singular Value Decomposition · Matrix Decomposition · Dimensionality Reduction
Source: http://datahacker.rs/009-the-singular-value-decompositionsvd-illustrated-in-python/