Model Training

concepts · 3 notes linked

Related: Large Language Models · Google Research · Hyperparameter Tuning · Deep Learning · Openai · Nvidia · Hugging Face · Microsoft

Notes

• ChatGPT and generative AI are booming, but the costs can be extraordinary — Compute economics of training and serving large language models
• GitHub - google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models. — Systematic hyperparameter tuning playbook from Google Brain researchers
• Technology Innovation Institute Open-Sourced Falcon LLMs: A New AI Model That Uses Only 75 Percent of GPT-3's Training Compute, 40 Percent of Chinchilla's, and 80 Percent of PaLM-62B's — TII open-sources Falcon-40B and Falcon-7B compute-efficient decoder LLMs