Principal Component Analysis with numpy

Abstraction: Implementing PCA via numpy covariance matrix and eigendecomposition

Key points:

• princomp() function: subtracts column means, computes covariance matrix, performs eigendecomposition via numpy.linalg.eig
• Returns three values: coeff (eigenvector matrix, p×p), score (projections into PC space), latent (eigenvalues of covariance matrix)
• First PC = direction of maximum variance; subsequent PCs are orthogonal to all prior ones
• Rank-deficient matrices (e.g., 3×4 rank-2): only first two components carry information; cumsum(latent)/sum(latent) shows variance explained
• Note: linalg.eig does not always return sorted eigenvalues — sorting required for canonical PCA ordering
• Inspired by MATLAB's princomp function from the statistics toolbox

Connections: Numpy · Principal Component Analysis · Dimensionality Reduction · Eigendecomposition

Source: http://glowingpython.blogspot.com/2011/07/principal-component-analysis-with-numpy.html