GitHub - jessevig/bertviz: BertViz: Visualize Attention in Transformer Models

Abstraction: Interactive Jupyter tool visualizing transformer attention weights

Key points:

• BertViz provides three views: head view (per-head attention), model view (all layers and heads at once), and neuron view (individual query/key vector neurons)
• Works with any Hugging Face model via a simple Python API; installable with pip install bertviz
• Extends the original Tensor2Tensor visualization by Llion Jones; published at ACL 2019 as "A Multiscale Visualization of Attention in the Transformer Model"
• Neuron view is limited to custom-patched BERT, GPT-2, and RoBERTa because it requires access to raw query/key vectors not exposed by the standard Hugging Face API
• Authors caution that attention weights illuminate architecture but do not directly explain model predictions; saliency methods (e.g., LIT, Ecco) are recommended for input-attribution tasks

Connections: Bertviz · Huggingface · Transformers · Attention Mechanism · Model Interpretability

Source: https://github.com/jessevig/bertviz