Mean Field Theory

concepts · 2 notes linked

Related: Matthias Bal · Attention Mechanisms · Transformers · Deep Equilibrium Models · Giorgio Parisi · David Sherrington · Spin Glass · Replica Method

Notes

• Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms — Attention as mean-field fixed-point response of Ising spin systems
• Spin glass - Wikipedia — Disordered magnetic system with frustrated interactions and non-ergodic phases