Module 1 Quiz - Categorical Foundations of ML

Question 1 of 6

A model trained on chest X-rays from one hospital fails to diagnose the same conditions when deployed at a different hospital with different imaging equipment. This is best described as:

Overfitting to training data
Covariate shift causing extrapolation failure
Underfitting due to insufficient model capacity
A labeling error in the test set

Correct. The input distribution has shifted (different equipment → different pixel statistics) while the underlying diagnostic task has not. This is covariate shift, a form of distribution shift that causes extrapolation failure when the model has no structural understanding of what makes an X-ray diagnostically meaningful across equipment variations.

Incorrect. The correct answer is B. The model may have fit the training data well and have sufficient capacity — the problem is that the input distribution has shifted (different imaging equipment produces different pixel statistics). This is covariate shift causing extrapolation failure.

Question 2 of 6

The manifold hypothesis, if true, implies which of the following about learning from natural data?

Models must be trained on exponentially more data as input dimension increases
The effective complexity of a learning problem is determined by the dimension of the data manifold, not the ambient dimension
All natural data is linearly separable in the ambient space
Neural networks automatically learn the correct manifold structure from examples

Correct. If data concentrates on a k-dimensional manifold in ℝ^d with k ≪ d, then the intrinsic complexity is k-dimensional. This is the basis for dimensionality reduction methods and for the argument that structural priors about the manifold can make high-dimensional problems tractable.

Incorrect. The correct answer is B. The manifold hypothesis says data lies on a low-dimensional manifold. The dimension of that manifold — not the ambient dimension — determines the effective complexity. This is why structural priors about manifold geometry can make high-dimensional problems tractable.

Question 3 of 6

Which of the following is an example of a structural constraint rather than a statistical regularizer?

L2 weight decay to prevent large parameter values
Dropout to prevent co-adaptation of neurons
Requiring predicted crystallographic defect types to belong to π₁(SO(3))
Early stopping based on validation loss

Correct. The fundamental group π₁(SO(3)) = ℤ₂ classifies topological defects in crystals. This is a mathematical theorem about the structure of the order parameter space — it cannot be violated by any physical system. A model that predicts defect types outside this group is provably wrong regardless of training performance. L2, dropout, and early stopping are all statistical regularizers with hyperparameters; they constrain the magnitude of the hypothesis but not its structural type.

Incorrect. The correct answer is C. L2, dropout, and early stopping are all statistical regularizers — they control the magnitude of the hypothesis space but do not eliminate structurally impossible models. Requiring predictions to belong to π₁(SO(3)) is a mathematical theorem about which defect types are physically possible; it eliminates entire connected components of wrong hypotheses.

Question 4 of 6

The No Free Lunch Theorem states that no learning algorithm outperforms random guessing when averaged over all possible target functions. The most precise response to this from the structural constraint perspective is:

The theorem is false and can be disproven by constructing a universal learning algorithm
The uniform distribution over target functions is the wrong prior for physical domains; the correct prior is concentrated on functions consistent with physical law
The theorem applies only to finite hypothesis spaces and is irrelevant for neural networks
More training data overcomes the NFL theorem in practice

Correct. The NFL Theorem is true but its premise — a uniform distribution over target functions — is empirically wrong for any physically structured domain. The space of physically realizable functions is a tiny, highly structured subset of all possible functions. Structural constraints encode this prior precisely: they say "the target function is in this subspace" rather than "the target function could be anything."

Incorrect. The correct answer is B. The NFL Theorem is mathematically valid. The structural response concedes the theorem but attacks its premise: the uniform distribution over target functions is not the right prior for physical domains. Physical law concentrates probability on a tiny, structured subset of all possible functions, and morphism structure encodes that concentration.

Question 5 of 6

In the course thesis, "the model is the category" means:

The model's parameters are stored as categorical data types in the implementation
The architecture of the neural network determines which category the model belongs to
Specifying the objects and morphisms of a category is the complete specification of the model's generalization behavior
Categories are used as a software design pattern for organizing model code

Correct. The claim is that once you have specified the objects (states, observations, configurations) and morphisms (valid transformations, structural relationships) of a category, the generalization behavior is fully determined — via Kan extensions for pointwise extension, and sheaf conditions for local-to-global consistency. There are no weights to tune and no architecture to design. The specification of the category IS the model.

Incorrect. The correct answer is C. "The model is the category" is a claim about generalization: specifying objects and morphisms fully determines how known values propagate to unseen states (Kan extension, Module 8) and when local consistency extends globally (sheaf condition, Module 11). The category replaces the weight vector as the fundamental modeling primitive.

Question 6 of 6

Which statement best characterizes the difference between a morphism and a regularizer as forms of inductive bias?

Regularizers are applied after training; morphisms are applied during training
Regularizers penalize but do not eliminate structurally wrong hypotheses; morphisms eliminate them by construction
Morphisms are a stronger form of regularizer that can be tuned like a hyperparameter
Regularizers work in high dimensions; morphisms only work in low-dimensional spaces

Correct. A regularizer (L1, L2, dropout) adds a penalty term to the loss that discourages certain hypotheses but does not make them unreachable. A structurally wrong model trained with any regularizer can still produce structurally wrong predictions — the regularizer just makes them less likely. A morphism, by contrast, eliminates structurally impossible transitions from the hypothesis space entirely: a model that cannot represent a physically impossible transformation cannot predict one, regardless of training pressure.

Incorrect. The correct answer is B. This is the key distinction of the course. Regularizers are soft constraints — they penalize wrong hypotheses but do not make them unreachable. Morphisms are hard constraints — they eliminate entire classes of structurally impossible hypotheses from the hypothesis space by construction. No training procedure, adversarial input, or distribution shift can push a categorically constrained model into a structurally impossible prediction.