latency reduction
Techniques aimed at minimizing the delay between input and output in AI systems, enhancing user experience and real-time processing capabilities.
- Accelerating Parallel Diffusion Model Serving with Residual Compression
- Adaptive Fission: Post-training Encoding for Low-latency Spike Neural Networks
- Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents
- ESCA: Enabling Seamless Codec Avatar Execution through Algorithm and Hardware Co-Optimization for Virtual Reality
- Fourier Token Merging: Understanding and Capitalizing Frequency Domain for Efficient Image Generation
- Parallel Scaling Law for Language Models
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
- SNAP: Low-Latency Test-Time Adaptation with Sparse Updates
- SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning