rotary position embedding

4 papers

• HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models
• Rope to Nope and Back Again: A New Hybrid Attention Strategy
• SALS: Sparse Attention in Latent Space for KV Cache Compression
• Tensor Product Attention Is All You Need