Llama Cpp

entities · 2 notes linked

Related: Local LLM Inference · GPT Oss · Whisper · Home Assistant · Openai · Proxmox · Mixture Of Experts · Quantization

Notes

• I'm running a 120B local LLM on 24GB of VRAM, and now it powers my smart home — Running gpt-oss-120b locally on 24GB VRAM via MoE
• LLMs-local — awesome platforms, tools, and resources for running LLMs locally — Awesome list for running LLMs locally