When people first browse local models, the file names can look like radio static: Q4_K_M, Q5_K_S, Q8_0, GGUF, instruct, coder, 32k, 128k.
Most of that noise is about one question: how much model quality can fit inside the memory you actually have?
What GGUF Is
GGUF is a model file format used by llama.cpp and many local tools built around it. Ollama, LM Studio, and other desktop-friendly runtimes often make GGUF models feel simple, even when the file underneath is doing a lot of work.
GGUF is the package. Quantization is the squeeze.
What Quantization Does
Quantization stores the model weights with fewer bits. That makes the file smaller and easier to run, but it can reduce quality. The trick is finding the point where the model is still good enough and finally fast enough.
Lower quant
Smaller file, less memory, faster start. More risk of weaker reasoning or rougher writing.
Higher quant
Larger file, more memory, usually better fidelity. Better when the hardware can breathe.
A Practical Rule
For a first test, Q4 is often the entry point. Q5 can be a nicer daily balance. Q8 is closer to the original model but needs more memory. Bigger is not automatically better if it makes every answer painfully slow.
The best local model is usually the one that fits comfortably, responds quickly, and is good enough for the task you repeat every day.
Skip the math: LLMFit scans your machine and lists every model that fits it, at each quantization, so the tradeoff table becomes a concrete answer for your hardware.