Post Training Quantization

concepts · 1 notes linked

Related: Llama · Meta · Model Quantization · Large Language Models

Notes

• The Trick to Make LLaMa Fit into Your Pocket: Meet OmniQuant, an AI Method that Bridges the Efficiency and Performance of LLMs — OmniQuant learnable post-training quantization achieves low-bit LLM compression efficiently