Chain Of Thought

concepts · 7 notes linked

Related: Large Language Models · Prompt Engineering · Google · Reinforcement Learning · Chatgpt · Palm · T5 · Knowledge Distillation

Notes

• AI Prompt Engineering Is Dead — Automated prompt optimization outperforms human prompt engineering
• Can 1B LLM Surpass 405B LLM? Optimizing Computation for Small LLMs to Outperform Larger Models — Test-time scaling lets small LLMs outperform much larger models
• Distilling step-by-step: Outperforming larger language models with less training — Rationale-based distillation trains small models to outperform 540B LLMs with far less data
• OpenAI Publishes GPT Prompt Engineering Guide — OpenAI's six-strategy guide for eliciting better GPT-4 responses
• Prompt Engineering Embraces New Technique Called Skeleton-Of-Thought — Skeleton-of-thought outlining prompt technique explained
• RAFT: Adapting Language Model to Domain Specific RAG — Fine-tuning recipe training LLMs to ignore distractor documents in domain RAG
• ReAct: Synergizing Reasoning and Acting in Language Models — Interleaved reasoning traces and external actions improve LLM agent reliability