GPT-2

entities · 3 notes linked

Related: Transformers · Openai · Large Language Models · Tokenization · GPT-3 · Mechanistic Interpretability · Superposition · Andrej Karpathy

Notes

• Once "too scary" to release, GPT-2 gets squeezed into an Excel spreadsheet — GPT-2 fully implemented in Excel spreadsheet for LLM education
• The Singular Value Decompositions of Transformer Weight Matrices — SVD of GPT-2 weight matrices reveals interpretable semantic directions
• Transformer Explainer: LLM Transformer Model Visually Explained — Interactive visual walkthrough of Transformer/GPT-2 architecture