← Course Overview ML2: Advanced Machine Learning
← Back to Week 4

Week 4: Vector Representations & Similarity Measures

Discussion Topics — Use these prompts for class discussion, online forums, or personal reflection.

Evaluation

1. The Granularity Dilemma in Recommender Systems

Streaming services like Spotify and Netflix must choose how finely to represent user preferences. A genre-level representation ("user likes Action films") is coarse but generalizes well, while a scene-level representation ("user engages when there is rapid editing") is precise but produces extreme sparsity and cold-start problems for new content.

  • How would you decide on the right level of granularity for a given recommender system? What signals or data would guide that decision?
  • Could a hierarchical vector — representing preferences at multiple levels of specificity simultaneously — solve the granularity problem? What are the tradeoffs?
  • When does over-generalization cause real harm to users — for instance, by erasing culturally specific preferences under a broad category?

Try: Ask students to sketch a two-level hierarchy (genre and sub-genre) for a domain they know, then vote on which level they would use for a recommender system and defend their choice in small groups.

Analysis

2. Cosine Similarity and the Erasure of Intensity

Cosine similarity measures the angle between vectors and is therefore insensitive to vector magnitude. This means a user who rated three items "mildly positive" and a user who rated the same three items "extremely positive" receive a perfect similarity score of 1.0, even though their intensity of preference is very different.

  • In which real-world applications does ignoring magnitude actually help — and in which does it hurt?
  • Could you design a hybrid similarity measure that captures both directional alignment and intensity difference? What would it look like mathematically?
  • How might magnitude differences reflect not just preference strength but also user behavior patterns (e.g., a user who rates everything highly versus one who uses the full scale)?

Try: Have students compute cosine similarity and Euclidean distance for two pairs of toy vectors by hand, then compare how each measure responds when one vector is scaled up by a factor of 3.

Synthesis/Creation

3. Manual Feature Engineering vs. Learned Representations: Who Decides What Matters?

Traditional survey-based feature engineering puts human experts in control of which dimensions are captured. Deep learning reverses this: the model learns what dimensions matter from data. Each approach embeds assumptions — either the researcher's or the dataset's — about what is meaningful.

  • What kinds of biases might be baked into manually engineered feature spaces that would not appear in learned representations, and vice versa?
  • If a learned representation discovers a feature dimension that correlates with race or socioeconomic status in a food-preference dataset, what ethical obligations does the developer have?
  • Is interpretability always desirable? Are there contexts where a black-box learned representation is preferable to an interpretable manual one?

Try: Split the class — one half designs a manual feature set for food preferences and the other brainstorms what a neural network might discover instead. Have groups compare their lists and discuss what each approach would miss.

Application

4. Sparse Representations and the Zero-Similarity Trap in Healthcare

Medical informatics frequently uses sparse binary vectors to represent patient histories — each dimension corresponds to a specific diagnosis code or medication. Two patients with very similar underlying conditions may have near-zero cosine similarity if their physicians used different but equivalent diagnostic codes.

  • What are the clinical consequences of a patient-similarity system failing to identify two patients as similar because of coding differences rather than medical differences?
  • How might ontologies (like SNOMED CT or ICD hierarchies) be incorporated into a vector representation to reduce spurious dissimilarity?
  • Should we prefer a system that produces explainable similarity scores (e.g., "similar because both have hypertension") over a black-box learned embedding, even if the learned embedding is more accurate?

Try: Present two fictional patient records coded differently for the same condition and ask students to compute their cosine similarity, then discuss how an ontology-aware approach would change the result.

Analysis

5. Factor Analysis and PCA: What Do the Latent Dimensions Actually Mean?

When PCA is applied to a food preference dataset, the first principal component might explain 35% of variance. Researchers often try to interpret this component by examining which original features load heavily onto it — perhaps it separates "health-conscious" from "indulgent" eaters. But this interpretation is post-hoc and potentially misleading.

  • How confident should we be in human-generated interpretations of PCA components? What can go wrong when we label a latent dimension and act on it?
  • PCA assumes linearity. What kinds of preference structures would be fundamentally undetectable by PCA but discoverable by a nonlinear method?
  • If you were building a food-preference product and PCA revealed an unexpected latent dimension, how would you decide whether it reflects a genuine user segment or a statistical artifact?

Try: Show students a real PCA biplot from a public dataset and run a class poll asking them to name the latent dimensions — then compare their labels and discuss how much agreement exists and why disagreement arises.

Discussion Facilitation Tips

  • Ground abstract vector concepts in concrete examples early — food preferences, movie ratings, or playlist data are immediately accessible and help students move from formula to intuition before engaging with edge cases.
  • When discussing cosine similarity versus Euclidean distance, use a live in-class calculation with small 2D or 3D vectors drawn on the board so students can see geometrically why magnitude is ignored by cosine similarity.
  • For the ethics discussions (especially topics 3 and 4), establish early whether the class is reasoning from a developer perspective, a regulator perspective, or an affected-user perspective — different framings produce very different and complementary insights.
  • Encourage students to challenge each other on the granularity tradeoff: push them to identify a specific domain where their preferred level of granularity would fail, which develops critical thinking about context-dependence in representation design.