Matthias Bal

entities · 1 notes linked

Related: Attention Mechanisms · Mean Field Theory · Transformers · Deep Equilibrium Models

Notes

• Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms — Attention as mean-field fixed-point response of Ising spin systems