kv cache
A memory optimization technique used in transformer architectures to store key-value pairs from previous computations, allowing models to efficiently retrieve relevant information without recomputing, improving inference speed.
- Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
- EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval
- KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
- KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
- PolarQuant: Leveraging Polar Transformation for Key Cache Quantization and Decoding Acceleration
- Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
- Sim-LLM: Optimizing LLM Inference at the Edge through Inter-Task KV Reuse
- UniGist: Towards General and Hardware-aligned Sequence-level Long Context Compression