Understanding matrices intuitively, part 1 - The Stata Blog

linear-algebramatricesvisualizationlinear-regressionstata

Abstraction: Visual grid-transformation method for intuiting matrix operations

Key points:

Matrices can be visualized by plotting how they transform a regular grid: y = Ax maps every point of the input grid to output, revealing stretch, compression, rotation, skew, or flip
A singular matrix squishes 2D space down to a line (or point); rank = number of remaining dimensions
Nearly singular matrices cause numeric instability: small differences in y blow up when computing x = A^-1 y
The linear regression formula b = (X'X)^-1 X'y can be read as: naive coefficients X'y corrected by the lens (X'X)^-1 which accounts for variable scale and correlation
X'X is the distortion due to data scale and correlations; (X'X)^-1 is the corrective lens mapping naive to true coefficients
Nonsquare matrices are equivalent in a deep sense to square matrices with zero rows/columns appended

Connections: Stata · Linear Algebra · Matrix Transformations · Linear Regression