Module 1 Readings: Why Structure Beats Statistics
Complete the required readings before lecture. Recommended readings provide additional depth and are referenced in discussion.
Required
-
Position PaperYann LeCun — 2022 — Read §1–3 (pages 1–12)Focus on: why LeCun argues current architectures fail at planning and reasoning, his diagnosis of the failure modes, and his proposed remedy (JEPA). We will return to this paper in Module 13 and evaluate it categorically.
-
Survey — SectionBronstein, Bruna, Cohen, Veličković — 2021 — Read §1 (Introduction, pages 1–8)Focus on: the "geometric principle" as an alternative to black-box learning, the blueprint for symmetry-aware architectures, and the claim that most successful deep learning can be understood through geometric structure.
Recommended
-
Blog PostChris Olah — 2014 — Accessible visual introduction to the manifold hypothesis and how neural networks deform manifolds
-
PaperHupkes et al. — 2020 — Empirical study showing that compositional structure must be encoded as bias, not learned from data
-
Classic PaperVon Luxburg — 2007 — §1–2 only; a clean example of geometric structure (graph Laplacian) encoding prior knowledge about cluster topology
-
Textbook — PreviewFong and Spivak — 2018 — Preface and §1.1 only; establishes the philosophical case for compositional thinking before the formalism
Reading Notes
When reading the LeCun paper, keep a running list of:
- Every claim about why current models fail (not just that they fail)
- Every proposed remedy and the implicit assumption it makes about what the problem is
- Anything that sounds like it might have a categorical formulation
This list will be your starting point for the Module 13 critical analysis project.