Ollama for Beginners: Install and Run Your First Model
You have heard that Ollama is the easiest way to run an AI model on your own machine, you have installed it, and now a blinking cursor is staring back at you. This guide walks through exactly what to type, in order, from a fresh install to a working chat — plus the handful of commands you will actually keep using and the errors that trip up almost everyone on day one.
No prior terminal experience needed. If you can copy and paste one line at a time, you can finish this.
What Ollama is (in one paragraph)
Ollama is a free, open-source program that downloads AI model files and runs them on your computer. It handles the messy parts for you: finding the right file, fitting it to your graphics card or CPU, and keeping it in memory between questions. You talk to it either in a terminal or through a local web address that other apps can call.
Think of it as a package manager plus a server for language models. You say "run this model," it fetches the model and gives you a prompt.
If you are still deciding whether to run models on your own hardware at all, the broader comparison in my guide to running an AI chatbot locally covers the alternatives and the hardware trade-offs. This article assumes you have already picked Ollama and want to get it working.
Step 1: Install Ollama
Download the installer from the official site, ollama.com. Nothing else is required — no account, no API key.
Windows: Run the downloaded installer. It adds the ollama command and starts a background service that launches with Windows. You will see a small llama icon in the system tray when it is running.
macOS: Open the downloaded app and drag it to Applications, then launch it once so it can set up the command-line tool. Apple Silicon Macs use the GPU automatically.
Linux: The official one-line install script is on the site. It installs the binary and sets up a systemd service, so Ollama starts on boot.
To confirm the install worked, open a terminal (on Windows, use PowerShell or Windows Terminal) and run:
ollama --version
A version number means you are ready. An error like "command not found" usually means the terminal was already open before you installed — close it and open a new one.
Step 2: Run your first model
This is the only command that really matters:
ollama run llama3.2
That one line does three things: downloads the model if you do not have it, loads it into memory, and drops you into a chat prompt. The first run takes a few minutes because of the download; after that it starts in seconds.
Type a question, press Enter, and you have a local AI. To leave the chat, type /bye or press Ctrl+D.
Why llama3.2 is a good first pick
It is small (a few gigabytes), it runs acceptably on a laptop without a dedicated graphics card, and it is good enough to tell whether your setup works. I’d always start small — reaching straight for a 70-billion-parameter model on a thin laptop is the single most common beginner mistake, and it may download for an hour and then crawl along at a word every few seconds.
A few other beginner-friendly names to try later, all in the same ollama run <name> format:
| Model | Good for | Rough size |
|---|---|---|
llama3.2 |
General chat, first test | Small |
gemma3 |
General chat, several sizes offered | Small to large |
qwen2.5 |
Chat and code, many size options | Small to large |
phi4 |
Reasoning on modest hardware | Medium |
llava |
Describing images you paste in | Medium |
nomic-embed-text |
Embeddings for search, not chat | Tiny |
Model names can take a tag after a colon to pick a size or precision — for example ollama run gemma3:1b for the smallest variant. If you leave the tag off, Ollama picks a sensible default. The model library on ollama.com lists every available tag and its file size, which is the number I’d compare against your free memory.
As a rule of thumb, a model needs roughly as much free memory as its download size, plus a little headroom. A 5 GB model on a machine with 8 GB of total RAM will technically load and then disappoint you.
Step 3: The Ollama commands you will actually use
You only need about seven. Run these in a normal terminal, not inside a chat session.
ollama list— shows every model you have downloaded and its size on disk. The first command to run when you forget what you have.ollama pull <model>— downloads a model without starting a chat. Handy before going offline.ollama run <model>— chat with a model (downloads it first if needed).ollama ps— shows which models are loaded in memory right now and whether they are on GPU or CPU. This is the fastest way to see if your graphics card is being used.ollama show <model>— prints the model's details: parameter count, context length, license, and its system prompt.ollama rm <model>— deletes a model to reclaim disk space. Model files add up fast.ollama serve— starts the server manually. You rarely need this on Windows or macOS, where it already runs in the background.
Useful commands inside a chat
Once you are at the chat prompt, lines starting with / are instructions to Ollama rather than messages to the model:
/?— list all available slash commands./set system "You are a terse assistant who answers in one sentence."— give the model a persistent personality for this session./set parameter num_ctx 8192— increase the context window so the model can hold more of a long conversation. Bigger windows use more memory./clear— wipe the conversation history without restarting the model./bye— exit.
That /set system command is the fun one. I’ve been active in chatbot communities since 2009, and changing a system prompt and watching the same model behave like a different assistant is still the cheapest lesson there is in how chatbot personality actually works — and it costs nothing to experiment with.
Step 4: Save your own setup with a Modelfile
Retyping a system prompt every session gets old. Ollama lets you bake your settings into a named model using a plain text file called a Modelfile.
Create a file named Modelfile (no extension) containing:
FROM llama3.2
PARAMETER temperature 0.3
SYSTEM "You are a technical editor. Rewrite text to be clearer and shorter. Never add new facts."
Then build and run it:
ollama create editor -f Modelfile
ollama run editor
You now have a model called editor that behaves consistently every time. temperature controls randomness: lower values (around 0.2 to 0.3) make output more predictable, higher values (0.8 and up) make it more creative. What I like about this is that creating a model this way does not duplicate the weights on disk, so it is cheap to do as often as you like.
Step 5: Using Ollama from your own code
Ollama runs a local web server on port 11434. Any program on your machine can talk to it, which is how desktop chat interfaces and coding assistants plug into it.
The quickest proof it is alive: visit http://localhost:11434 in a browser and you should see "Ollama is running."
To send a prompt from the command line:
curl http://localhost:11434/api/generate -d "{\"model\": \"llama3.2\", \"prompt\": \"Explain port numbers in two sentences.\", \"stream\": false}"
There is also an OpenAI-compatible endpoint at http://localhost:11434/v1/chat/completions, which means many tools and code libraries written for commercial APIs can be pointed at your local Ollama by changing the base URL and passing any placeholder API key. That compatibility is a big part of why Ollama caught on — it slots into software that was never written with local models in mind.
Common Ollama problems and how to fix them
| Symptom | Likely cause | Fix |
|---|---|---|
| Answers come one word at a time | Running on CPU, not GPU | Check ollama ps; try a smaller model or update your GPU drivers |
| "Out of memory" or a crash on load | Model too large for your RAM or VRAM | Use a smaller tag, such as :3b instead of :8b |
| "command not found" | Terminal was open before install | Close and reopen the terminal |
| Port 11434 already in use | Ollama is already running in the background | Use the running instance, or stop the service before ollama serve |
| Download stalls partway | Network interruption | Re-run ollama pull; it resumes rather than starting over |
| Disk filling up | Old models never deleted | ollama list, then ollama rm the ones you do not use |
| Model forgets earlier messages | Context window too small | Raise it with /set parameter num_ctx |
Two environment variables are worth knowing when the defaults do not suit you. OLLAMA_MODELS changes where model files are stored, which matters when your system drive is small and you would rather keep models on a second disk. OLLAMA_HOST changes the address and port the server listens on. I’d be deliberate with that second one — binding to every network interface exposes an unauthenticated AI server to your whole network, so only do it on a network you trust.
Frequently Asked Questions
Is Ollama free?
Yes. Ollama itself is open source and free to download and use, and the models in its library are open-weight models you download at no cost. The only real expenses are disk space and electricity.
Where does Ollama store downloaded models?
In a hidden .ollama folder inside your user directory by default — for example C:\Users\<you>\.ollama\models on Windows. Set the OLLAMA_MODELS environment variable to move that folder to another drive, then restart Ollama.
Can Ollama run without a graphics card?
Yes. Ollama falls back to your CPU and system RAM, and small models stay usable that way. Expect noticeably slower responses, and stick to the smallest model tags for a reasonable experience.
What is the difference between ollama run and ollama pull?
pull only downloads the model and exits. run downloads it if needed, then loads it and opens a chat. Use pull to prepare models in advance, especially before you lose internet access.
How do I update a model to a newer version?
Run ollama pull <model> again. If the publisher has released an updated version under the same tag, the new files download and replace the old ones; if nothing has changed, the command finishes almost immediately.
Does Ollama send my prompts anywhere?
Your conversations are processed on your own machine and are not sent to Ollama's servers. Ollama does use the internet to download models and check for updates, so you can pull the models you want in advance and then work entirely offline.
The takeaway
Ollama turns running a local AI model into two steps: install it, then type ollama run llama3.2. Everything after that — tags, Modelfiles, the local API, context windows — is refinement you can pick up as you need it.
Your next step is a small one: run ollama list, pick a second model roughly the same size as your first, and ask both the same question. I’d trust that comparison over any spec sheet when it comes to choosing models.