Self Attention

concepts · 4 notes linked

Related: Transformers · Large Language Models · Attention Mechanism · Google · Openai · GPT-3 · Computer Vision · GPT-2

Notes

• GitHub - sainathadapa/attention-primer-pytorch — PyTorch toy tasks demonstrating attention mechanisms from Vaswani et al.
• Transformer Explainer: LLM Transformer Model Visually Explained — Interactive visual walkthrough of Transformer/GPT-2 architecture
• Why multi-head self attention works: math, intuitions and 10+1 hidden insights | AI Summer — Mathematical and intuitive deep-dive into multi-head self-attention mechanisms
• Will Transformers Take Over Artificial Intelligence? | Quanta Magazine — Transformers expanding from NLP to vision, generative, and multimodal AI tasks