Week 9: From Supervised to Generative Learning — Knowledge Check

Question 1

A) Both are discriminative — they both operate on digital data
B) The spam filter is generative; the image generator is discriminative
C) The spam filter is discriminative (deciding between classes); the image generator is generative (creating new data)
D) Both are generative — any model that produces an output is generative

Question 2

A) Medical images are higher resolution, making storage costs prohibitive even before annotation begins.
B) Accurate annotation requires expert knowledge (e.g., board-certified radiologists for pixel-wise segmentation), creating a bottleneck that cannot easily be crowdsourced or automated.
C) Medical institutions do not allow their data to be used for machine learning research, regardless of annotation cost.
D) Medical imaging uses different file formats that are incompatible with standard deep learning frameworks, slowing the annotation pipeline.

Question 3

A) The data is compressed into a lower-dimensional latent code that captures its essential structure.
B) Gaussian noise is progressively added to the data in small steps until the original signal is completely destroyed, yielding approximately pure noise.
C) The model learns to transform random noise into realistic data by applying a series of learned refinement steps in the forward direction.
D) The data is discretized into tokens at each step, converting continuous images into a sequence of categorical values.

Question 4

A) It means the model only needs to see the very first step to predict what the final noisy image looks like
B) It means each step can be computed from just the previous one, which massively simplifies the math and makes the process tractable
C) It means the same noise pattern is reused at every step to save computation
D) It means the model can skip most steps and only process every other one

Question 5

A) The final clean image directly, in one step
B) The noise present in the current noisy image, so it can be subtracted to recover a slightly cleaner image
C) A quality score for how realistic the current noisy image looks
D) Which specific training image the current noisy version originally came from

Question 6

A) It allows the model to use a different set of weights for each time step, increasing the total parameter count.
B) It tells the model what noise level it is currently working with, allowing a single network to apply appropriate denoising behavior at every stage from high-noise to near-clean.
C) It synchronizes the denoising steps across different samples in a batch to ensure consistent generation quality.
D) It enables the model to skip unnecessary denoising steps when the input is already close to a clean sample.

Question 7

A) The supervisory signal comes from human annotators who label the masked tokens with their correct meanings.
B) The supervisory signal is derived from the data itself — the model is trained to predict the original tokens that were masked out, using the surrounding context as supervision without any manual labels.
C) The supervisory signal is provided by a separate teacher model that generates target distributions over the vocabulary for each masked position.
D) The supervisory signal comes from a fixed linguistic grammar that specifies the correct word at each masked position based on syntactic rules.

Question 8

A) A positive pair consists of an image and its correct class label; the model is trained to assign high probability to the correct label.
B) A positive pair consists of two different augmented views of the same data point; the model is trained to produce similar representations for positive pairs and dissimilar representations for views from different data points.
C) A positive pair consists of two images from the same class in the dataset; the model is trained to classify them identically.
D) A positive pair consists of a noisy image and its clean counterpart; the model is trained to reconstruct the clean version from the noisy input.

Question 9

A) Both approaches require a separately trained discriminator network to provide a learning signal to the generative model.
B) Diffusion model denoising can be viewed as a form of masked prediction where Gaussian noise serves as the corruption, and the model learns to reconstruct the original data — creating its own supervision signal from the data structure without labels.
C) Both approaches model the data distribution P(X) explicitly using maximum likelihood estimation over the training set.
D) Diffusion models and self-supervised models both rely on contrastive objectives that push noise embeddings away from clean data embeddings in representation space.

Question 10

A) Similar images are always brighter, so the model learns to detect brightness and lighting
B) By learning what makes two different views of the same thing "the same," the model is forced to capture real content rather than surface details
C) The model memorizes which specific images are paired together during training
D) Contrastive learning is just supervised classification without labels