Module 13: Categorical Machine Learning: Synthesis

This final module draws all threads together. Major ML architectures are re-read through the categorical lens developed over the course: geometric deep learning as symmetry by construction, neural networks as functors, and JEPA as an empirical approximation to a Renormalization Group fixed point. The module addresses the central debate of the field — LeCun's partition function intractability argument — and presents the sheaf-condition resolution: local consistency along morphisms replaces global normalization. The module runs as a research seminar, closing with student presentations of original contributions.

Learning Objectives

Explain equivariance as a consequence of functoriality rather than a property learned from data.
Identify the morphism gap between JEPA's learned representations and a true RG fixed-point representation.
State LeCun's Z-intractability argument precisely and present the sheaf-condition resolution.
Critique a major ML architecture using the categorical vocabulary of the course.
Present original research connecting a specific ML system to its categorical ideal or failure mode.
Identify at least two open problems at the boundary of category theory and machine learning.

Materials

Key Concepts

Equivariance by Construction: If a group G acts on a category C as automorphisms, any functor that factors through the quotient C/G is automatically G-equivariant; no data augmentation or additional loss term required.
Morphism Gap: The structural deficiency of a system that approximates a categorical ideal without preserving morphisms; JEPA's encoder discards "irrelevant" information by an empirical criterion, not by an RG flow equation.
Sheaf Resolution: LeCun's Z is intractable as a global integral; replacing it with local consistency checks along morphisms (the sheaf condition) gives an exact and tractable criterion for membership on the data manifold.
Neural Network as Functor: A parameterized map between categories of activation patterns that preserves compositionality; the weights parameterize the natural transformation family, not the category itself.
RG Fixed-Point Representation: A representation invariant under further coarse-graining; the "meaning" of an input; what a well-trained encoder should converge to; JEPA approximates this without the mathematical guarantee.
Open Problem: A well-posed research question at the frontier; examples include: Is mean-field variational inference the pointwise Lan of the variational family inclusion? Does sheaf cohomology of the loss landscape predict generalization?

Central Concepts from Prerequisites

All prior modules
Graduate ML: energy-based models, VAEs, transformers, GNNs, JEPA
Research skills: reading primary literature, identifying open problems