An Applied Mathematician Strengthens AI With Pure Math | Quanta Magazine

Abstraction: Applying topology and algebraic geometry to explain and improve neural networks

Key points:

• Lek-Heng Lim (U Chicago, 2022 Guggenheim fellow): cat vs. non-cat image classification modeled as disentangling two intertwined topological manifolds — deep network layers progressively simplify their topology, measurable via persistent homology
• Persistent homology estimates the "holes" (homology) of a continuous manifold from only discrete point samples (e.g., image pixels), providing a formal tool to track how data structure changes through network layers
• Lim and Lai disproved a ~10-year-old conjecture: sampling parameters without replacement is not universally better than with replacement — proven using the noncommutative Positivstellensatz from algebraic geometry
• GPT-4 rumored at 1T-100T parameters; random subset sampling (with vs. without replacement) is a fundamental operation at that scale
• Historical argument: the pure/applied math split is only ~80 years old; Gauss, Hilbert, and Von Neumann made no such distinction

Connections: Lek Heng Lim · University Of Chicago · Machine Learning · Topology · Neural Networks · Pure Mathematics

Source: https://www.quantamagazine.org/an-applied-mathematician-strengthens-ai-with-pure-math-20230301/