Who's #1?

Abstraction: Critical book review of rating and ranking methods lacking statistical grounding

Key points:

• Review of "Who's #1? The Science of Rating and Ranking" by Langville and Meyer; reviewer criticizes that none of the 8+ methods covered are tied to a probabilistic model, loss function, or underlying "truth"
• Methods presented include Markov chain ranking, Elo rating, and others — all criticized as "conventions" rather than representations of ground truth
• Elo rating system (used in chess) is described as "epitome of simple elegance" but the reviewer notes its high arbitrariness; the system goal is consensus convention, not truth
• Markov approach constructs a transition matrix solely to derive a stationary distribution — the reviewer finds the underlying structure non-dynamical and thus forced
• Key criticism: "Since there is no validation for one ordering against another, we can see no rationality in proposing any of those, except to set a convention"
• A statistics-based treatment of rankings (using models, loss functions, selection theory) is argued to be more principled than the book provides

Connections: Ranking Algorithms · Rating Systems · Markov Chains

Source: https://xianblog.wordpress.com/2012/05/02/whos-1/