Ollama and four other local model runners, compared
I ran the same three jobs through five ways of serving an open-weight model on one homelab machine. Ollama is where most people should start; vLLM is where a team ends up.
Sample post. The structure, the licences and the feature columns are real; the setup times and speeds are illustrations, not measurements.
What I tested
Each runner served the same model to the same three jobs: a coding agent editing a small TypeScript repository, a script summarising 200 support emails, and three people chatting through one shared front end at once.
I installed each one from its own documentation on a clean VM with the GPU passed through, timed the setup from the first command to the first answer, and kept the notes on what the docs left out.
- Setup: from a fresh Debian VM to a working API call.
- API: whether an OpenAI-style client works with only the base URL changed.
- Load: how each one behaves when three requests arrive together.
- Upkeep: updating the runner and swapping a model without a rebuild.
Setting each one up
Ollama took one install script and ollama run with a model name. LM Studio is a desktop app, so on a headless VM it needs its command-line tool and a little patience. llama.cpp wants a build with the right GPU flags, and the server flags decide how many requests run in parallel.
vLLM installs with pip into a Python environment and expects weights in the Hugging Face layout. It was the slowest to get going and the best under load. LocalAI pulls a container image and maps models to backends through YAML files, which is flexible and one more place for a typo.
curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1:8b
curl http://localhost:11434/v1/chat/completions -d @request.jsonComparison table
The columns that decided the verdict. Setup time and speed are illustrations for this draft; the real post prints measured ones.
| Runner | Licence | Cost | Interface | OpenAI-style API | Model formats | Several users at once | Setup time, illustrative | Tokens a second, illustrative | Verdict |
|---|---|---|---|---|---|---|---|---|---|
 Ollama | MIT | Free | Command line and a local API | Yes | GGUF, pulled from its own library or Hugging Face | Queues requests; a few in parallel | 6 min | 71 | Start here |
 LM Studio | Proprietary, free to use | Free | Desktop app with a local server | Yes | GGUF; MLX on Apple silicon | One person at a desk | 4 min | 68 | Best for trying models by hand |
| llama.cpp server | MIT | Free | Command line and an HTTP server | Yes | GGUF | Parallel slots, set at start | 25 min | 74 | For people who want every knob |
 vLLM | Apache 2.0 | Free | Python server with an HTTP API | Yes | Hugging Face weights, AWQ and GPTQ | Continuous batching for many users | 40 min | 62 | Pick it for a team |
| LocalAI | MIT | Free | HTTP API with a web page | Yes | GGUF and others, through backends | Depends on the backend | 30 min | 66 | Wide, but more parts to break |
Where each one fits
Ollama is the shortest path from nothing to a working local API. Its model library hides the file formats, and most tools that speak to OpenAI work with it by changing one URL.
LM Studio is the friendliest way to try models by hand on a laptop. Its licence is not open source, which matters if you plan to ship it inside something.
llama.cpp is the engine under several of the others. Run it directly when you want control over every setting and do not mind reading flags.
vLLM is built for serving many requests at once. If a whole team will share one GPU, start here and accept the longer setup.
LocalAI covers the most ground, including images and speech, at the cost of more moving parts.
Verdict
Install Ollama first. It will tell you within an afternoon whether local models are good enough for your work. If they are, and more than a few people need them, move the serving to vLLM and keep Ollama on the laptops.