Cosine similarity, Pearson correlation, and OLS coefficients

Abstraction: Unified inner-product view of cosine similarity, correlation, and OLS

Key points:

• Cosine similarity = normalized inner product: CosSim(x,y) = <x,y> / (||x|| ||y||); bounded [-1,1], not shift-invariant
• Pearson correlation = CosSim(x - x_bar, y - y_bar): centering makes it shift-invariant, still bounded [-1,1]
• OLS coefficient (no intercept) = <x,y> / ||x||^2: one-sided normalization only by x, not symmetric, not shift-invariant
• OLS with intercept = <x - x_bar, y> / ||x - x_bar||^2: shift-invariant for x; subtly also invariant to shifts of y
• OLS on standardized inputs equals Pearson correlation
• Summary: Cosine = normalized inner product; Correlation = centered cosine; OLS = cosine with one-sided normalization

Connections: Cosine Similarity · Correlation · Linear Regression · Inner Product

Source: http://brenocon.com/blog/2012/03/cosine-similarity-pearson-correlation-and-ols-coefficients/