Model Efficiency

concepts · 1 notes linked

Related: Tinyllama · Meta · Large Language Models · Scaling Laws

Notes

• Meet TinyLlama: A Small AI Model that Aims to Pretrain a 1.1B Llama Model on 3 Trillion Tokens — Small 1.1B LLM pre-trained on 3 trillion tokens