throughput improvement

3 papers

• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
• EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
• What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions