RAFT: Adapting Language Model to Domain Specific RAG

Abstraction: Fine-tuning recipe training LLMs to ignore distractor documents in domain RAG

Key points:

• RAFT (Retrieval Augmented Fine Tuning) is a fine-tuning strategy for domain-specific open-book QA with RAG
• Trains the model on data that includes distractor documents alongside oracle documents, forcing it to identify and quote only relevant passages
• Uses chain-of-thought answers with explicit quote markers (##begin_quote##/##end_quote##) to reduce hallucination
• Some training examples deliberately omit the oracle document, compelling the model to memorize domain knowledge
• Built on Llama-2-7B; trained on Azure AI Studio; evaluated on PubMed, HotpotQA, and Gorilla API benchmarks, outperforming baselines
• Key insight: smaller fine-tuned models can match or exceed general LLMs on domain-specific QA

Connections: Uc Berkeley · Microsoft · Meta · Llama 2 · Retrieval Augmented Generation · Fine Tuning · Chain Of Thought

Source: https://gorilla.cs.berkeley.edu/blogs/9_raft.html