Machine Learning Research Blog

Abstraction: Francis Bach's blog on optimization theory and ML fundamentals

Key points:

• Research blog by Francis Bach (INRIA / ENS Paris) covering theoretical ML and optimization
• Core recurring topics: gradient-based optimization, quadratic functions, Hessian eigensubspaces, convergence rates, condition numbers
• Covers scaling laws: characterizes optimality gaps as Theta(k^-p) depending on problem structure and dimension
• Explores spectral properties of kernel matrices and their eigenvalue decay, relevant to statistical and algorithmic properties
• Discusses Nesterov acceleration, Jensen's inequality, and convex vs non-convex optimization
• Author published book "Learning Theory from First Principles" (hard copies received late 2023)

Connections: Francis Bach · Optimization · Kernel Methods · Convex Optimization

Source: https://francisbach.com/