Radial Attention: $\mathcal O(n \log n)$ Sparse Attention for Long Video Generation

Shuo Yang (University of California, Berkeley) · Ion Stoica (UC Berkeley) · Song Han (MIT, NVIDIA) · Lvmin Zhang (Stanford University) · Muyang Li (Massachusetts Institute of Technology) · Maneesh Agrawala (Stanford University) · Kurt Keutzer (EECS, UC Berkeley) · XINGYANG LI (Shanghai Jiao Tong University) · Tianle Cai (Princeton University) · Haocheng Xi (University of California, Berkeley) · Yujun Lin (NVIDIA) · Songlin Yang (MIT & Thinking Machines Lab) · Jinbo Hu (Shanghai Jiaotong University) · Kelly Peng (Stanford University)

$\mathcal{o}(n \log n)$ complexity$\mathcal{o}(n^2)$ dense attentionattention maskcompute densitydiffusion models inference acceleration linear attentionlora-based fine-tuningpost-softmax attentionradial attentionsparse attention mechanismspatiotemporal energy decaytemporal distance video generation video quality

Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training and inference on long videos prohibitively expensive. In this paper, we identify a phenomenon we term Spatiotemporal Energy Decay in video diffusion models: post-softmax attention scores diminish as spatial and temporal distance between tokens increase, akin to the physical decay of signal or waves over space and time in nature. Motivated by this, we propose Radial Attention, a scalable sparse attention mechanism with $\mathcal{O}(n \log n)$ complexity that translates energy decay into exponentially decaying compute density, which is significantly more efficient than standard $\mathcal{O}(n^2)$ dense attention and more expressive than linear attention. Specifically, Radial Attention employs a simple, static attention mask where each token attends to spatially nearby tokens, with the attention window size shrinking with temporal distance. Moreover, it allows pre-trained video diffusion models to extend their generation length with efficient LoRA-based fine-tuning. Extensive experiments show that \method maintains video quality across Wan2.1-14B, HunyuanVideo, and Mochi 1, achieving up to a 1.9× speedup over the original dense attention. With minimal tuning, it enables video generation up to 4× longer while reducing training costs by up to 4.4× compared to direct fine-tuning and accelerating inference by up to 3.7× compared to dense attention inference. Code is released at https://github.com/mit-han-lab/radial-attention.