throughput
Throughput in AI refers to the amount of data processed or the number of operations performed by an AI system within a specific time frame. High throughput is important for real-time applications.
- Accelerating Diffusion LLMs via Adaptive Parallel Decoding
- Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
- Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
- ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
- First Attentions Last: Better Exploiting First Attentions for Efficient Parallel Training
- GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projection
- Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
- NestedFP: High-Performance, Memory-Efficient Dual-Precision Floating Point Support for LLMs
- Private Training Large-scale Models with Efficient DP-SGD
- Quartet: Native FP4 Training Can Be Optimal for Large Language Models
- SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
- Traversal Verification for Speculative Tree Decoding
- Tri-MARF: A Tri-Modal Multi-Agent Responsive Framework for Comprehensive 3D Object Annotation
- Zebra-Llama: Towards Extremely Efficient Hybrid Models