Deploy DeepSeek R1 Locally (2026): Real VRAM Math + Setup

Every tutorial for DeepSeek R1 says the same thing: install Ollama, type ollama run deepseek-r1, done. Then you get a CUDA out of memory crash, or worse – the model silently runs on CPU at one token per second and you don’t notice for an hour.

The problem isn’t the tool. It’s that nobody shows the actual memory math. This guide covers a real deployment of R1’s distilled models via Ollama – VRAM numbers that include the KV cache, the storage trap on Windows, and the Modelfile knobs the official docs bury. The underlying architecture is described in the DeepSeek-R1 paper (arXiv 2501.12948) if you want to go deeper on why the distillation approach works.

What you’re actually deploying

R1 shipped in January 2025 under an MIT license – free for commercial use, weights on Hugging Face. DeepSeek has since released V3-0324, R1-0528, V3.1, and V3.2, but R1 remains the reference open-weight reasoning model and the one Ollama’s deepseek-r1 tag family points at.

The full model is 671B parameters (Mixture-of-Experts, 37B active per token). You are almost certainly not running that. What you’re running is a distill – a smaller dense model (Qwen or Llama base) trained to imitate R1’s chain-of-thought. Available sizes: 1.5B, 7B, 8B, 14B, 32B, 70B. All pulled through the same ollama pull deepseek-r1:<size> command.

Hardware requirements – with the KV cache included

The VRAM numbers most guides show only cover model weights. The KV cache is a separate allocation that scales with context length – and it’s usually what pushes you over the edge on a tight card. Honest numbers at Q4_K_M (Ollama’s default as of 2026), with an 8K context budget added:

*8B figure measured; larger models estimated proportionally. Sources: willitrunai, gigagpu, praveentechworld.

The KV cache math for the 8B distill: 2 × 32 layers × 8 KV heads × 128 head dim × 2 bytes = 128 KB per token. At num_ctx 8192 that’s 1 GB you won’t see in the model card. Double the context, double the cache.

The benchmark jump pattern is worth pausing on. 7B → 14B is +14 points on AIME 2024. 14B → 32B? Only +3 points (69.7% → 72.6%). If you’re on a 16 GB card, the 14B is the honest sweet spot – the 32B costs double the VRAM for a gain that’s nearly invisible on most real tasks. On an RTX 4090, the 14B distill runs at around 58 tok/s; the 32B drops to roughly 28 tok/s (benchmarks from runaihome, 2026).

Install and pull – the whole sequence

Set the storage path before pulling anything. Ollama defaults to ~/.ollama on Unix and C:Users<user>.ollama on Windows – a 32B pull is around 20 GB and will silently fill your OS drive if you skip this step. Linux and macOS get the one-liner from ollama.com/download; Windows uses the installer on the same page.

# Linux / macOS - install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Move the model cache off your OS drive FIRST
# Linux/macOS:
export OLLAMA_MODELS=/mnt/data/ollama
# Windows (PowerShell as admin, then restart Ollama):
[Environment]::SetEnvironmentVariable("OLLAMA_MODELS", "D:ollama", "Machine")

# Verify Ollama is up
ollama --version

# Pull the distill that matches your VRAM
ollama pull deepseek-r1:14b

# Run it
ollama run deepseek-r1:14b

The first prompt will be slow – Ollama loads weights into VRAM on first use. Subsequent prompts hit the cached model.

First-run configuration

Defaults are not tuned for reasoning. DeepSeek’s own recommendation is temperature 0.6 and top_p 0.95 – the standard 0.8 temperature makes R1 wander on logic tasks. A Modelfile bakes this in permanently:

cat > Modelfile <<'EOF'
FROM deepseek-r1:14b
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER num_ctx 8192
EOF

ollama create r1-tuned -f Modelfile
ollama run r1-tuned

num_ctx is the one you’ll fight with. Bigger context = larger KV cache = less headroom. On a 16 GB card running the 14B distill, 8192 is safe. Push to 32K and you’ll OOM on long conversations.

Keep-alive trap: Ollama holds models in VRAM for 5 minutes after the last request. If you run two models back-to-back, the first one is still sitting in memory when the second loads. Fix: send {"model": "deepseek-r1:14b", "keep_alive": 0} to /api/generate to unload immediately.

Here’s a question worth sitting with before you scale up: what does “local” actually mean for your use case? If you need reproducible outputs – same model, same quantization, same results tomorrow – Ollama’s tag system won’t give you that. The deepseek-r1:14b tag has pointed to different quantization revisions across 2025 and 2026 as DeepSeek shipped updates. For experiments that need to be repeatable, you want a pinned GGUF blob from Hugging Face, not a floating tag. For everyone else, the tag is fine.

Verifying it works (not just that it loaded)

A model that loaded is not a model that’s running on the GPU. Three checks:

1. ollama ps – the PROCESSOR column is the one that matters. “100% CPU” means Ollama silently fell back because the GPU allocation failed. Target: RTX 4090 should hit ~58 tok/s on the 14B distill. If you’re seeing 3-5 tok/s, you’re on CPU.
2. nvidia-smi – during a prompt, GPU utilization should spike to 90%+ and VRAM usage should match the table. Flat GPU = CPU inference.
3. Ask it a logic or coding problem and confirm the visible <think> block appears before the final answer. That’s R1’s reasoning chain – intentional, not a rendering bug. No <think> block usually means you’re on a non-R1 tag.

Errors you’ll actually hit

cudaMalloc failed: out of memory – model plus KV cache exceeds VRAM. Two fixes: drop num_ctx to 4096 first (free, instant). If still failing, set OLLAMA_NUM_GPU_LAYERS=20 to offload some layers to CPU RAM – slower, but it runs.

model request too large for system – see Ollama issue #8734. Someone tried the 671B on a single 180 GB GPU. Ollama loads all MoE weights regardless of the 37B-active architecture; the error log showed a request for 391 GiB. No fix on consumer hardware – use a distill or the hosted API.

GPU never lights up. Ollama fell back to CPU – check with ollama ps first. Then try CUDA_VISIBLE_DEVICES=0 before starting the server. Still nothing? Run OLLAMA_DEBUG=1 ollama serve and look for the layer allocation report in the logs.

Responses stop mid-sentence. Context limit hit. Reasoning models burn tokens on the <think> chain before answering – an 8K window fills faster than you’d expect with a complex prompt. Raise num_ctx if VRAM allows, or trim your input.

Upgrading, and getting rid of it

# Upgrade to whatever :14b currently points at
ollama pull deepseek-r1:14b

# See what's stored locally and how big it is
ollama list

# Remove a specific model
ollama rm deepseek-r1:14b

# Full uninstall (Linux)
sudo systemctl stop ollama
sudo rm /usr/local/bin/ollama
sudo rm -rf /usr/share/ollama
rm -rf $OLLAMA_MODELS # or ~/.ollama if you never moved it

One caveat: deepseek-r1:14b does not pin the underlying quantization revision. If reproducibility matters, pull the specific GGUF from Hugging Face (search deepseek-ai/DeepSeek-R1-Distill-Qwen-14B or bartowski’s mirrors) and register it via a Modelfile with an explicit blob hash. The official DeepSeek-V3 GitHub has the model card, architecture notes, and inference code for reference.

If you want to push further – vLLM for higher throughput, LM Studio for a GUI, or Open WebUI in front of Ollama – start with the 14B distill, confirm your token rate matches the published benchmarks, then scale up.

FAQ

Should I run the 32B or the 14B if I have a 24 GB card?

14B. The AIME jump is only ~3 points, and you get roughly twice the token rate. The 32B makes sense if you’re specifically benchmarking model quality – not for everyday use.

Can DeepSeek R1 generate images or handle audio?

No. R1 is text-only – no vision encoder, no audio input, as of mid-2026. This trips people up because ChatGPT’s o1 (which R1 is often compared against) also has no image generation natively, but the ChatGPT product wraps it in a multimodal shell. If you need image generation alongside local reasoning, you’ll need a separate model like Stable Diffusion running in parallel. R1 can describe what an image should look like; it cannot produce one.

Is the local version as good as chat.deepseek.com?

Only if you run the full 671B – which almost nobody does locally. The distills are impressive for their size, but they’re approximations. For the closest match to the hosted experience, use the DeepSeek API.

Next step: run ollama pull deepseek-r1:7b right now, prompt it with a logic puzzle you already know the answer to, and check ollama ps to confirm the GPU is actually being used. If both work, scale up to the 14B and set your Modelfile.