Week 4: Vector Representations & Similarity Measures

Course Overview

This week explores vector representations and similarity measures through practical examples and theoretical foundations.

Learning Objectives

• Understand vector representations and their challenges
• Master cosine similarity calculations and interpretation
• Compare manual vs learned feature engineering
• Appreciate the role of representation granularity

Topics Covered

1. Food Preference Vector Example

• Representing diets as vectors
  • Binary feature vectors
  • Frequency-based vectors
  • Rating-based vectors
• Introduction to cosine similarity
  • Vector operations review
  • Geometric interpretation
  • Similarity calculations

2. The Problem of Sparse Representations

• Highly specific features
  • Individual ingredient level
  • Brand-specific items
  • Regional variations
• Challenges with sparse vectors
  • Zero similarity problem
  • Curse of dimensionality
  • Missing relationships

3. Over-generalization Problems

• Too broad categories
  • "Protein" vs "Fish" vs "Wild-caught Salmon"
  • Loss of meaningful distinctions
• Impact on similarity measures
  • False equivalences
  • Lost nuances
  • Meaningless similarities

4. Traditional Survey Design Approaches

• Feature engineering through survey design
  • Likert scales
  • Categorical hierarchies
  • Controlled vocabularies
• Statistical methods
  • Factor analysis
  • Principal component analysis
  • Traditional dimensionality reduction

5. Learning Representations

• Deep learning approach
  • From raw data to learned features
  • Automatic feature extraction
  • Hierarchical representations
• Benefits of learned representations
  • Capturing natural relationships
  • Finding optimal granularity
  • Preserving important distinctions
• Measuring success
  • Cosine similarity in learned space
  • Validation through known relationships
  • Discovering new patterns