communication overhead
Communication overhead refers to the additional time and resources required for data exchange in distributed AI systems, which can impact the overall efficiency and speed of collaborative learning processes.
- ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
- Accelerating Parallel Diffusion Model Serving with Residual Compression
- Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
- Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism
- CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment
- DictPFL: Efficient and Private Federated Learning on Encrypted Gradients
- Diffusion Federated Dataset
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts
- First Attentions Last: Better Exploiting First Attentions for Efficient Parallel Training
- Hierachical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
- Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training
- NAVIX: Scaling MiniGrid Environments with JAX
- NormFit: A Lightweight Solution for Few-Shot Federated Learning with Non-IID Data
- Tight analyses of first-order methods with error feedback
- Towards Robust Parameter-Efficient Fine-Tuning for Federated Learning
- ZeCO: Zero-Communication Overhead Sequence Parallelism for Linear Attention