token selection

3 papers

• Accurate KV Cache Eviction via Anchor Direction Projection for Efficient LLM Inference
• Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference
• Limitations of Normalization in Attention