← Back to Week 2

Week 2: Neural Networks & Backpropagation

Knowledge Check — Select an answer to see immediate feedback.

Questions 10

Question 1

When information flows forward through a neural network, each layer receives some input, transforms it, and passes the result to the next layer. What is the input to any given hidden layer?

A) The raw training data, always passed directly from the original inputs
B) The weight matrix that will be applied at that layer
C) The activated outputs from the previous layer
D) The error signal computed during the backward pass

Question 2

Backpropagation relies on the chain rule of calculus. What is the chain rule's role in this context?

A) It computes the forward pass activations layer by layer
B) It allows the gradient of the loss with respect to early-layer weights to be computed by multiplying local gradients through each layer
C) It determines the learning rate automatically based on curvature
D) It ensures that gradients are always positive during training

Question 3

During training, the model takes a step in a direction that should reduce its error. What happens if those steps are made too large?

A) The model trains faster and finds a better solution
B) The model's updates overshoot the target, causing the loss to bounce around or get worse
C) The gradients disappear and the network stops learning
D) The model immediately settles on the perfect solution

Question 4

What is the key difference between stochastic gradient descent (SGD) and mini-batch gradient descent?

A) SGD uses the entire dataset per update; mini-batch uses a random subset
B) SGD updates parameters using a single random training example; mini-batch uses a small fixed-size subset of examples
C) SGD uses a fixed learning rate; mini-batch automatically schedules the learning rate
D) Mini-batch gradient descent always produces worse estimates than full-batch

Question 5

Different tasks need different ways to measure how wrong a model's predictions are. For which type of task is Mean Squared Error (MSE) the most natural choice?

A) Classifying images into categories like "cat" or "dog"
B) Deciding whether an email is spam or not
C) Predicting a continuous number, like a house price or tomorrow's temperature
D) Generating a translation of a sentence into another language

Question 6

A model predicts that an image is a cat with 99% confidence — and it is correct. What does the loss for that prediction look like?

A) The loss is large, because the model should never be that confident
B) The loss is close to zero — a confident correct prediction is exactly what we want
C) The loss is exactly 1.0 regardless of confidence level
D) The loss goes negative, pulling the overall total down

Question 7

The vanishing gradient problem occurs during backpropagation. What is the primary cause of this issue in deep networks?

A) The learning rate is set too low
B) Gradient values become very small as they are multiplied through many layers, especially when activation function derivatives are less than 1, making early-layer weights learn very slowly
C) The batch size is too large, diluting the gradient signal
D) The weight matrices are initialized to zero

Question 8

During training, a neural network needs to figure out how much each weight contributed to the overall error so it can adjust them all. Why is working backward through the network (backpropagation) efficient for this?

A) It only needs to look at the final output layer, ignoring all hidden layers
B) It computes how the loss depends on every single weight in one backward pass, regardless of how many weights there are
C) It stores the entire training dataset in GPU memory to calculate gradients
D) It is used only during testing, not during training

Question 9

One advantage of mini-batch gradient descent over full-batch gradient descent is that the stochasticity in gradient estimates can help escape local minima and saddle points. Why?

A) Random noise in the gradient introduces directional perturbations that can push parameters out of flat or locally optimal regions of the loss landscape
B) Mini-batch gradients are always more accurate than full-batch gradients
C) Full-batch gradient descent automatically avoids saddle points due to its use of the complete data
D) Mini-batches increase the learning rate dynamically to escape flat regions

Question 10

Computational graphs provide a structured representation of the operations in a neural network. Why are they essential for backpropagation?

A) They store the training data more efficiently than arrays
B) They explicitly record which operations produced which values, enabling the chain rule to be applied systematically in reverse to compute all gradients
C) They replace the need for a loss function by encoding the target directly
D) They ensure that only the last layer's gradients are computed, saving computation