Kaiyue Wen

3 papers Stanford University

• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
• PaTH Attention: Position Encoding via Accumulating Householder Transformations