R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

Junjie Hu (The Chinese University of Hong Kong, Shenzhen) · Animashree Anandkumar (Caltech) · Zefan Cai (University of Wisconsin - Madison) · Wen Xiao (Microsoft) · Hanshi Sun (ByteDance Seed) · cheng Luo (caltech) · Yikai Zhang (Department of Computer Science, University of Wisconsin - Madison) · Ke Wan (University of California San Diego) · Yucheng Li (University of Surrey & Microsoft) · Yeyang Zhou (University of California San Diego) · Li-Wen Chang (ByteDance Seed) · Jiuxiang Gu (Adobe Systems) · Zhen Dong (UCSB NVIDIA) · Abedelkadir Asi (Microsoft)
baseline comparisoncache reductionchain-of-thought reasoningexperimental resultsinference optimizationkey-value cachekv cache compressionmathematical reasoning datasetsmemory savingperformance preservationreasoning failuresreasoning modelsredundant tokensself-reflectionspeedup

Reasoning models have demonstrated impressive performance in self-reflection and chain-of-thought reasoning. However, they often produce excessively long outputs, leading to prohibitively large key-value (KV) caches during inference. While chain-of-thought inference significantly improves performance on complex reasoning tasks, it can also lead to reasoning failures when deployed with existing KV cache compression approaches. To address this, we propose Redundancy-aware KV Cache Compression for Reasoning models (R-KV), a novel method specifically targeting redundant tokens in reasoning models. Our method preserves nearly 100% of the full KV cache performance using only 10% of the KV cache, substantially outperforming existing KV cache baselines, which reach only 60% of the performance. Remarkably, R-KV even achieves 105% of full KV cache performance with 38% of the KV cache. This KV-cache reduction also leads to a 50% memory saving and a 2x speedup over standard chain-of-thought reasoning inference. Experimental results show that R-KV consistently outperforms existing KV cache compression baselines across two mathematical reasoning datasets.