attention scores
Attention scores are numerical values that indicate the relevance of specific input elements when processing sequences, enabling models like transformers to focus on pertinent information and improve performance on tasks like language translation.
- Attention (as Discrete-Time Markov) Chains
- AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
- Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization
- DePass: Unified Feature Attributing by Simple Decomposed Forward Pass
- FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language Models
- KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments
- On the Role of Hidden States of Modern Hopfield Network in Transformer
- SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs
- SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
- VORTA: Efficient Video Diffusion via Routing Sparse Attention