Local LLM performance usually comes down to a triangle: model size, context length, and speed. You can push one corner, but the other corners will feel it.
VRAM is not just where the model lives. It is also where the conversation starts to take up space.
Model Size
A larger model can understand more, write better, and make fewer strange mistakes. But if it barely fits, it may run slowly, spill into system RAM, or leave no room for a useful context window.
Context Length
Context is how much text the model can consider at once. Long context is useful for documents, codebases, transcripts, and agents. But a huge context window is not free. It uses memory and can slow the session down.
A 128k context window is powerful only when the hardware and the task can afford it.
Small and fast
Good for quick replies, summaries, classification, drafts, and everyday local assistants.
Large and deep
Better for harder reasoning and long documents, but usually slower and more demanding.
The Daily Driver Test
Ask the same real task across two or three models: one small, one medium, one ambitious. Time the answers. Read them side by side. The winner is often not the largest model. It is the model you will tolerate using all day.
For self-hosting, comfort matters: enough quality, enough speed, enough context, and no constant fighting with memory.
Shortcut: LLMFit shows all capable models for your hardware based on what is actually available, ranked by fit, which collapses this whole triangle into one scan.