AI Workbench

Category: Terms Map

Plain-language explanations of the vocabulary around self-hosted LLMs.

  • Terms That Unlock Self-hosted LLMs

    Self-hosted LLMs come with a lot of vocabulary. Most of it sounds harder than it is because the words arrive before the mental picture.

    Once the core terms click, the whole stack becomes easier to navigate.

    The Model Words

    Parameters are the learned weights inside the model. More parameters can mean more capability, but also more memory. Instruct usually means the model has been tuned to follow directions. Base means it is closer to the raw pretrained model and usually less friendly for chat.

    The Memory Words

    Context window is how much text the model can consider at once. Tokens are the pieces of text the model reads and writes. VRAM is GPU memory, and it becomes one of the main limits when running models locally.

    Tokens are the meter. Context is the room. VRAM is the floor space.

    The File Words

    GGUF

    A local-friendly model file format commonly used with llama.cpp-based tools.

    Quantization

    A way to shrink model weights so larger models can fit on smaller hardware.

    The System Words

    Runtime is the software that runs the model. Embedding is a numeric representation of meaning. RAG is the pattern where your documents are searched first, then relevant pieces are sent to the model.

    You do not need to master the whole vocabulary at once. Learn the terms that explain the next problem in front of you.

  • Start Here: Self-hosted LLMs

    Self-hosting an LLM means running the model on hardware you control: a laptop, workstation, home server, VPS with a GPU, or a machine reached over a private network.

    The goal is not to replace every cloud model. The goal is to know when local control is the better tool.

    A local model is less like a magic service and more like an engine you keep in your own workshop.

    Why People Run Models Locally

    The obvious reason is privacy. Prompts, documents, transcripts, notes, and business data can stay on your own machine. That matters when the work is sensitive, unfinished, or simply nobody else’s business.

    The second reason is control. You can choose the model, keep a stable version, test different quantizations, connect it to local tools, and build habits around one setup instead of chasing every new API release.

    Cloud Model vs Local Model

    Cloud

    Usually stronger, easier to start, and better for heavy reasoning, big multimodal tasks, and production APIs.

    Local

    Private, repeatable, cheaper at small scale, and good for drafting, coding help, search over notes, classification, and experiments.

    The Simple Stack

    A beginner setup can be very small: install Ollama or LM Studio, download a model, open a chat UI, and start testing. A more serious setup adds Open WebUI, a private network such as Tailscale, a document index, monitoring, and backups.

    Start with one small model that runs comfortably. A fast model you actually use teaches more than a giant model that barely fits in memory.