← Course Overview ML2: Advanced Machine Learning

Week 9: From Supervised to Generative Learning

Videos Textbook Colab Knowledge Check Discussion

Course Generated Slides: GANs

1. Beyond Supervised Learning

  • Limitations of labeled data
    • Cost of annotation
    • Expert knowledge requirements
    • Scale limitations
  • The generative alternative
    • Learning data distributions
    • Self-supervised learning
    • Implicit vs explicit modeling

2. Diffusion Models: Core Concepts

  • Forward process
    • Noise scheduling
    • Gradual information destruction
    • Markov chains
  • Reverse process
    • Denoising steps
    • Score matching
    • Time conditioning
  • Applications and use cases
  • Implementation considerations

3. Self-Supervised Learning

  • Masked prediction tasks
    • BERT-style masking
    • Image patch prediction
    • Corruption and reconstruction
  • Contrastive learning approaches
    • Positive/negative pairs
    • SimCLR approach
    • Momentum contrast
  • Connection to diffusion models
    • Denoising as self-supervision
    • Learning without labels
    • Representation quality

Required Reading

  • "Denoising Diffusion Probabilistic Models" (Ho et al.)
  • "Understanding Diffusion Models: A Unified Perspective"

Learning Objectives

  • Understand the transition from supervised to generative approaches
  • Master the principles of diffusion models
  • Grasp self-supervised learning techniques
  • Connect diffusion concepts across modalities
← Week 8: Convolutional Neural Networks Week 10: Introduction to Large Language Models →