latency

3 papers

• Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability
• LoRO: Real-Time on-Device Secure Inference for LLMs via TEE-Based Low Rank Obfuscation
• SeerAttention: Self-distilled Attention Gating for Efficient Long-context Prefilling