← Course Overview ML2: Advanced Machine Learning

Week 12: Retrieval Augmented Generation (RAG)

Videos Textbook Colab Knowledge Check Discussion

1. Foundations of RAG

  • Beyond model knowledge
    • Limitations of pretrained knowledge
    • Need for current information
    • Domain-specific knowledge
  • RAG architecture
    • Retriever component
    • Generator component
    • Integration patterns
  • Vector databases
    • Embedding storage
    • Similarity search
    • Index management

2. Building RAG Systems

  • Document processing
    • Text chunking strategies
    • Metadata extraction
    • Embedding generation
  • Retrieval strategies
    • Dense retrieval
    • Hybrid search
    • Re-ranking approaches
  • Context integration
    • Prompt construction
    • Context window management
    • Response synthesis

3. RAG Implementation

  • Vector store options
    • Pinecone
    • Weaviate
    • FAISS
  • System design
    • Caching strategies
    • Batching operations
    • Performance optimization
  • Quality control
    • Relevance assessment
    • Source attribution
    • Fact verification

Required Reading

  • "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"
  • "Vector Databases: From Embeddings to Applications"

Learning Objectives

  • Understand RAG architecture and components
  • Master document processing for retrieval
  • Implement efficient RAG systems
  • Evaluate retrieval quality and performance
← Week 11: Practical LLM Integration & API Development Week 13: Evaluating LLMs - Metrics and Methods →