← Back to Week 5

Week 5: Autoencoders & Embeddings

Knowledge Check — Select an answer to see immediate feedback.

Questions 10

Question 1

In a standard autoencoder, what is the role of the bottleneck (latent) layer?

A) To apply dropout regularization to prevent overfitting
B) To force the network to learn a compressed representation of the input
C) To store the reconstruction loss for backpropagation
D) To apply batch normalization across the full feature space

Question 2

An undercomplete autoencoder is defined as one where:

A) The decoder is smaller than the encoder
B) The latent dimension is smaller than the input dimension
C) The network has fewer layers than a standard deep network
D) The training loss is not fully minimized

Question 3

In a denoising autoencoder (DAE), what is the key modification to the standard training procedure?

A) The loss function is replaced with KL divergence
B) Corrupted inputs are fed to the network, but the target output is the original clean data
C) The encoder and decoder weights are shared (tied weights)
D) The latent space is constrained to a Gaussian distribution

Question 4

A Variational Autoencoder (VAE) has two goals at the same time. One is to reconstruct the input accurately. What is the second goal?

A) To make the encoder as small as possible to save memory
B) To keep the compressed representation in a well-organized, structured space so that new samples can be generated from it
C) To make the decoder run faster at inference time
D) To count how many neurons are active during encoding

Question 5

A VAE needs to randomly sample from the latent space during training. Why is this a problem for the learning process?

A) Random sampling produces too many numbers and slows down training
B) You cannot compute gradients through a random sampling step, so the model cannot learn which direction to improve
C) Random sampling always produces the same output, making training repetitive
D) Sampling requires the decoder to run twice as many calculations

Question 6

In anomaly detection using autoencoders, what serves as the anomaly score?

A) The magnitude of the latent vector
B) The reconstruction error — anomalous inputs that differ from training data tend to reconstruct poorly
C) The KL divergence from the standard normal prior
D) The number of active neurons in the bottleneck layer

Question 7

What is "disentanglement" in the context of latent representations?

A) The process of separating the encoder and decoder into independent networks
B) A property where different latent dimensions correspond to independent, interpretable factors of variation in the data
C) Removing correlations between training examples during batching
D) Ensuring the reconstruction loss and KL loss do not interfere with each other

Question 8

A sparse autoencoder is trained with an extra penalty in addition to reconstruction quality. What does this penalty encourage the network to do?

A) Use as many neurons as possible to represent each input as richly as possible
B) Keep most neurons inactive for any given input, so only a few fire at once
C) Force the decoder to have identical weights to the encoder
D) Reduce the learning rate automatically whenever reconstruction loss stops improving

Question 9

If you take two images encoded by a VAE and gradually blend their compressed representations, you get a smooth transition between them. Why is this smoother in a VAE than in a regular autoencoder?

A) VAEs use a larger compressed representation, leaving more room for smooth transitions
B) VAEs are trained to keep their compressed space structured and continuous, so any point between two valid encodings also decodes to something meaningful
C) VAEs always train on larger datasets, giving them better generalization
D) Regular autoencoders cannot compress images at all

Question 10

An autoencoder is trained only on medical scans from healthy patients. It is then shown a scan that contains a tumor it has never seen before. What would you expect to happen?

A) It would correctly identify the tumor because medical autoencoders always detect anomalies
B) It would reconstruct a healthy-looking version poorly, producing a high reconstruction error that flags the anomaly
C) It would refuse to process the scan because the tumor was not in its training data
D) It would produce a completely random output since it has never seen a tumor