← Back to Week 6

Week 6: From Autoencoders to Embeddings

Knowledge Check — Select an answer to see immediate feedback.

Questions 10

Question 1

In Word2Vec's Skip-gram model, the training objective is to:

A) Predict the target word given the average of its surrounding context words
B) Predict the surrounding context words given the target word
C) Minimize the KL divergence between word distributions
D) Reconstruct the full sentence from a compressed sentence embedding

Question 2

The famous word embedding analogy "king - man + woman = queen" demonstrates which property of word vectors?

A) Semantic similarity is measured by cosine similarity between individual word vectors
B) Relational structure is encoded as consistent geometric offsets, enabling vector arithmetic to solve analogies
C) The embedding space is discrete, with words occupying fixed grid positions
D) Gender information is stored in the magnitude of word vectors, not their direction

Question 3

Negative sampling in Word2Vec training is used primarily to:

A) Generate hard training examples that improve robustness
B) Reduce the computational cost of the softmax by only updating a small number of word vectors per step
C) Prevent the model from memorizing frequent words
D) Ensure that the embedding vectors remain orthogonal to each other

Question 4

How does GloVe (Global Vectors for Word Representation) differ from Word2Vec in its approach to learning embeddings?

A) GloVe uses a recurrent neural network, while Word2Vec uses a feedforward network
B) GloVe learns from global word co-occurrence statistics across the entire corpus, while Word2Vec uses local context windows
C) GloVe produces contextual embeddings that change based on sentence, while Word2Vec produces static embeddings
D) GloVe represents words as bags of character n-grams, while Word2Vec treats each word as atomic

Question 5

FastText improves over Word2Vec primarily by:

A) Using a transformer architecture with self-attention
B) Representing each word as a bag of character n-grams, enabling embeddings for out-of-vocabulary and rare words
C) Training on multilingual corpora to produce cross-lingual embeddings
D) Replacing the softmax output with a contrastive loss for faster convergence

Question 6

What is the key distinction between contextual embeddings (like BERT) and static embeddings (like Word2Vec)?

A) Static embeddings have higher dimensionality than contextual embeddings
B) In contextual embeddings, each word has a different representation depending on its surrounding context; static embeddings assign one fixed vector per word
C) Contextual embeddings can only be used for classification tasks, not for similarity search
D) Static embeddings are produced by transformers, while contextual embeddings use RNNs

Question 7

t-SNE is often used to visualize word embeddings. What does t-SNE do to make visualization possible?

A) It selects the two dimensions with the highest variance, similar to PCA
B) It uses a non-linear dimensionality reduction that minimizes KL divergence between pairwise similarity distributions in high- and low-dimensional spaces
C) It converts embedding vectors into 2D coordinates by computing their average pairwise cosine similarity
D) It clusters words into groups and then places group centroids in 2D space

Question 8

Fine-tuning pre-trained word embeddings for a downstream task means:

A) Selecting a subset of embedding dimensions most relevant to the task
B) Initializing the embedding layer with pre-trained weights and then updating those weights via backpropagation during task-specific training
C) Freezing the embedding weights and training only the downstream model layers
D) Re-running Word2Vec on a new corpus specific to the target domain before task training

Question 9

The "cold start problem" in recommendation systems using user-item embeddings refers to:

A) The slow convergence of embedding training at the beginning of the optimization process
B) The difficulty of generating meaningful recommendations for new users or items with little or no interaction history
C) The tendency for embeddings to degrade in quality when the corpus grows very large
D) The computational cost of initializing a large embedding lookup table from scratch

Question 10

Bias detection using word embeddings works by:

A) Checking whether the embedding vocabulary contains offensive words
B) Analyzing geometric relationships between word vectors — for example, measuring whether profession words are closer to "man" or "woman" vectors
C) Comparing embedding-based predictions to human-annotated ground truth labels
D) Monitoring the reconstruction error of an autoencoder trained on sensitive text