Transformers Explained Visually (Part 2): How it works, step-by-step

Abstraction: Step-by-step internal data flow through the Transformer architecture

Key points:

• Positional encoding uses interleaved sine (even indices) and cosine (odd indices) curves; added to embedding vectors; shape (samples, seq_length, embedding_size) preserved throughout
• Encoder stack: Multi-head Self-attention → Feed-forward, each with residual skip-connections and layer normalization
• Decoder stack: Masked Self-attention → Encoder-Decoder attention (Q from decoder, K/V from encoder output) → Feed-forward, each with residuals
• Decoder masking during training prevents "peeking" at future target words (teacher forcing scenario)
• Padding masking in encoder ensures padding tokens contribute zero to attention scores
• Output layer: Linear projection to vocabulary size → Softmax → argmax to select output word per position

Connections: Transformers · Attention Mechanism · Positional Encoding

Source: https://towardsdatascience.com/transformers-explained-visually-part-2-how-it-works-step-by-step-b49fa4a64f34