Scaling Laws

concepts · 3 notes linked

Related: Meta · Large Language Models · Tinyllama · Model Efficiency · Yandex · Recommender Systems · Transformers · Reinforcement Learning

Notes

• AI critic Gary Marcus: Meta's LeCun is finally coming around to the things I said years ago — Gary Marcus argues deep learning alone cannot achieve general intelligence
• Meet TinyLlama: A Small AI Model that Aims to Pretrain a 1.1B Llama Model on 3 Trillion Tokens — Small 1.1B LLM pre-trained on 3 trillion tokens
• Scaling Recommender Transformers to a Billion Parameters | Towards Data Science — ARGUS autoregressive recommender transformer scaled to one billion parameters