Deep Equilibrium Models

concepts · 1 notes linked

Related: Matthias Bal · Attention Mechanisms · Mean Field Theory · Transformers

Notes

• Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms — Attention as mean-field fixed-point response of Ising spin systems