Songlin Yang

4 papers MIT & Thinking Machines Lab

• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
• PaTH Attention: Position Encoding via Accumulating Householder Transformations
• Radial Attention: $\mathcal O(n \log n)$ Sparse Attention for Long Video Generation