Information Theory

concepts · 10 notes linked

Related: Claude Shannon · Entropy · Kl Divergence · Combinatorics · Harry Nyquist · Sampling Theorem · Signal Processing · Aliasing

Notes

• 100 Years of Computer Science — Curated timeline of ten landmark computer science papers 1923-2014
• Algorithm precisely quantifies flow of information in complex networks — TE-PWS algorithm exactly computes transfer entropy in complex nonlinear networks
• Information Theory Finds the Best Wordle Starting Words — Using Shannon entropy to optimize Wordle opening word selection
• Intuitive Explanation of the Kullback-Leibler Divergence — KL divergence measures extra bits lost approximating one probability distribution with another
• Kinetic proofreading - Wikipedia — Biochemical irreversible-step error-correction mechanism achieving accuracy beyond thermodynamic equilibrium limits
• Kullback-Leibler divergence - Wikipedia — Asymmetric measure of difference between two probability distributions
• Nyquist–Shannon sampling theorem - Wikipedia — Fundamental theorem defining minimum sampling rate for lossless reconstruction
• Position weight matrix - Wikipedia — Matrix representation of biological sequence motifs using position-specific log-odds scores
• The Secret Math Behind Mind-Reading Magic Tricks | Quanta Magazine — Mathematical structures — de Bruijn sequences and logic deduction — underlying magic tricks
• f-divergence - Wikipedia — General family of probability distribution divergences via convex function