How to Run Llama Locally with Ollama 0.34.2

The #1 mistake when people try to run Llama locally isn’t a bad install command. It’s pulling a model that doesn’t fit free RAM or VRAM, watching the download finish, then hitting model requires more system memory – or a crawl so slow it feels broken. Size the box first. Then install Ollama.

One binary. A model registry. API on 11434. That’s Ollama for local Llama (and other open weights). This write-up pins Ollama 0.34.2 – latest stable around mid-September 2026; 0.34.3 was still RC on the GitHub releases page. Specs, install paste-blocks, verify, the errors that actually show up, upgrade traps, cleanup.

System requirements before you touch the installer

Ollama itself is light. The model is the bill.

Windows 10+. Modern Linux the install script still supports. macOS – Apple Silicon if you care about Metal/MLX. Disk: model file plus a few GB headroom; small Llama tags land ~1-5 GB, big ones tens of GB.

NVIDIA: nvidia-smi must already work. AMD on Linux: host ROCm has to match what Ollama ships (ROCm 7 libs in current builds). Older drivers hang GPU discovery, then silently fall back to CPU – hours lost if you don’t check logs. No GPU? Fine. Single-digit tokens/sec on 7B-class CPU-only is normal, not a broken install.

Think of VRAM like a workbench: the whole quantized model wants to sit on it, then the KV cache grows with context. A 4.7 GB file does not mean “5 GB free is enough.”

Official download source

Only use these:

• Site / installers: ollama.com/download
• Docs: docs.ollama.com
• Source / releases: github.com/ollama/ollama/releases
• Docker Hub image: ollama/ollama

Skip random mirrors. Script + signed Windows/macOS packages are the supported path.

Install Ollama 0.34.2 step by step

Official one-liner (or GUI) first. Docker second.

Linux

curl -fsSL https://ollama.com/install.sh | sh

Binary, systemd unit when available, ollama on PATH. Want to inspect? Save the script, read it, then run.

Manual tarball (amd64) from the Linux docs – upgrading? Wipe old libs first or you debug ghost mismatches for an afternoon:

sudo rm -rf /usr/lib/ollama
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst 
 | sudo tar x -C /usr
ollama serve

AMD GPU extras: extract ollama-linux-amd64-rocm.tar.zst the same way. ARM64 has its own tarball. Pin with OLLAMA_VERSION=0.34.2 on the install script when you need the exact build.

macOS

curl -fsSL https://ollama.com/install.sh | sh

Or the .dmg from the download page → Applications → launch once so the CLI symlink lands. Apple Silicon picks up Metal/MLX in current builds without extra flags.

Windows

irm https://ollama.com/install.ps1 | iex

Or OllamaSetup.exe. Lands under %LOCALAPPDATA%ProgramsOllama, user PATH, background service. Close the terminal and open a new one – classic “ollama is not recognized” is almost always a stale PATH session.

Docker (brief)

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

NVIDIA: Container Toolkit first, then --gpus=all. AMD: image ollama/ollama:rocm plus /dev/kfd and /dev/dri. Models via docker exec -it ollama ollama ....

Pro tip: On Linux upgrades, forgetting sudo rm -rf /usr/lib/ollama before the new extract is how you get mysterious library skew. Do the rm first.

First-time configuration and running Llama

0.34.2 walks a first-run screen when you just type ollama (sign-in for cloud bits, or stay local). Pure offline Llama? Skip cloud.

Server up? Linux service often auto-starts; otherwise ollama serve or open the desktop app. Then pick a tag against free memory – not the marketing parameter count – pull, chat.

# Tight machines (~2 GB class download)
ollama pull llama3.2:3b
ollama run llama3.2:3b

# Usual 16 GB RAM / 8-12 GB VRAM pick (~4.7 GB)
ollama pull llama3.1:8b
ollama run llama3.1:8b

Chat is plain text; /bye exits. Library tags and quants move – check ollama.com/library before a multi-GB pull (sizes as listed there may change). Default library builds are typically Q4-class GGUF for consumer boxes.

Env knobs after the default path works: OLLAMA_HOST, OLLAMA_MODELS, OLLAMA_KEEP_ALIVE, context overrides. Not before.

Verify the install works

ollama -v
# or
ollama --version

curl -s http://localhost:11434/api/version
ollama list
ollama run llama3.2:3b "Reply with exactly: ok"

Expect 0.34.x in the version string, JSON from the API, the model in list, a short reply from run. Linux service weird? systemctl status ollama and journalctl -u ollama -n 50 – that pair beats random forum threads. While something’s loaded, ollama ps shows GPU layers vs CPU spill.

Common install errors and fixes

The catch is almost never “Ollama is broken.” It’s port, memory, drivers, or tmp mounts.

“could not connect” / connection refused on 11434 – nothing listening, or two things fighting. Start the app/service, or ollama serve. Desktop app already bound 11434? Don’t launch a second serve. Listeners: lsof -i :11434 (macOS/Linux) or netstat -ano | findstr 11434 (Windows).

“model requires more system memory (X GiB) than is available (Y GiB)” – sizing table ignored. Kill heavy apps, drop to llama3.2:1b/:3b, or a smaller quant if the library offers one. Fat num_ctx makes the failure show up faster.

GPU invisible / everything on CPU – NVIDIA: nvidia-smi, sometimes sudo nvidia-modprobe -u, reload nvidia_uvm, current driver. Docker: prove docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi before blaming the image. AMD Linux: user in video/render; host driver in the ROCm 7 class – older stacks timeout discovery and fall back (official troubleshooting notes this mismatch explicitly).

Linux tmp noexec – Ollama drops short-lived executables under tmp. Mount with noexec? Set OLLAMA_TMPDIR to a writable exec-capable path (often under /usr/share/ollama), restart the service.

Windows: ollama not recognized – new terminal after install; confirm %LOCALAPPDATA%ProgramsOllama on user PATH. Very old Win10 builds mangled terminal control codes – 22H2+ behaves.

Still stuck? OLLAMA_DEBUG=1 plus journalctl or app logs.

Upgrade and uninstall

Upgrade: same install script again on Linux/macOS. Windows: installer/script again. Docker: docker pull ollama/ollama:latest, recreate container (named volume keeps models). Pin: curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.34.2 sh.

Uninstall Linux (official docs pattern):

sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm -r $(which ollama | tr 'bin' 'lib') # libraries
sudo rm $(which ollama)
sudo userdel ollama
sudo groupdel ollama
sudo rm -r /usr/share/ollama
# models often also live under ~/.ollama - delete deliberately:
rm -rf ~/.ollama

macOS: quit app, remove /Applications/Ollama.app, sudo rm /usr/local/bin/ollama, then rm -rf ~/.ollama (plus optional Library caches) if models should go too.

Windows: Settings → Apps → Uninstall Ollama (or Inno under LocalAppData). Blobs stay in %USERPROFILE%.ollama until that folder dies. Current scripts also honor $env:OLLAMA_UNINSTALL=1; irm https://ollama.com/install.ps1 | iex.

App gone ≠ disk free. Decide on models on purpose.

Next: one model that fits free memory, the verify curl, then point a coding agent or OpenAI-compatible client at http://localhost:11434. UI later (Open WebUI and friends) only after the engine answers.

Ever notice how local-LLM guides sell “one command” and skip the afternoon you lose to ROCm group membership or a noexec /tmp? That’s the gap this checklist is for.

FAQ

Do I need a GPU to run Llama with Ollama?

No. CPU-only works. It’s slower. NVIDIA CUDA, AMD ROCm (Linux), or Apple Metal/MLX is what makes 7B-8B feel interactive.

Which Llama tag should I pull first on a laptop with 16 GB RAM?

llama3.2:3b first – prove pull + run + API. Still have headroom after OS and browser? Try llama3.1:8b. If ollama ps shows a hard CPU spill on a “small” tag, cut context before you blame the installer. Confirm download size on the model page; library tags change.

Why does uninstall still leave huge folders?

Installers remove the app and service. Model blobs live under ~/.ollama (Windows: %USERPROFILE%.ollama) on purpose so a reinstall can reuse them. Easy to miss when you’re reclaiming an SSD. ollama list, back up what you care about, then delete the directory. Docker users hit the same trap with the named volume.