Deploy DeepSeek-V3 Locally: Honest Install Guide (2026)

The endpoint you’re aiming for looks like this: curl http://localhost:8000/v1/chat/completions returns a JSON body with real tokens generated by DeepSeek-V3 on your own hardware. Everything below walks backward from that goal.

When people call DeepSeek-V3 the best open source AI model, they usually mean the 671B-parameter beast published under MIT license – a model that circulating benchmark reports at its release credited with outperforming most closed competitors on math and code. What they rarely mention: almost nobody reading a tutorial can actually run it. This guide is honest about what you’ll actually deploy.

What you’re actually installing (and which version)

The name “DeepSeek-V3” is a moving target. The lineage matters because copy-pasting install commands from a January 2025 blog post will give you a model that’s been superseded three times.

The GitHub repo tagged deepseek-ai/DeepSeek-V3 only has a single release, v1.0.0, marked archival. Actual current work lives in deepseek-ai/DeepSeek-V3.2-Exp. A tutorial pointing you at the V3 repo for weights is out of date.

Think of it like a band that keeps touring under the same name after three lineup changes. The original DeepSeek-V3 repo is the archived first album – technically the right name, practically not what anyone’s listening to anymore. The actual ongoing work has moved, quietly, and most tutorials never caught up.

System requirements – the honest version

Here’s the trap. MoE sparsity saves you compute, not memory. The DeepSeek-V3 technical report (arXiv:2412.19437) confirms 671B total parameters with 37B activated per token – but all 671B must sit in VRAM even though only 37B fire per forward pass. As of late 2025, gigagpu’s VRAM analysis pegs the minimum at 8x RTX 6000 Pro 96GB to hold the full model.

Official system constraints from the repo docs are strict: Linux only, Python 3.10, no macOS or Windows support. That’s for the reference implementation. Most of us won’t use it.

The realistic split:

• Full V3 (671B, FP8/BF16): multi-node GPU cluster. Rent it, don’t build it.
• Quantized V3 via llama.cpp/Unsloth: heavily quantized builds shrink the model significantly, but you still need substantial combined VRAM and RAM plus a fast NVMe.
• Distills (deepseek-r1-distill-* on Ollama): what you probably actually want. 8B fits an 8GB GPU, 32B fits a 24GB card at Q4_K_M.

The Ollama deepseek-v3 tag people paste around is not the 671B model. It’s a much smaller variant that shares branding. Say that out loud before you spend a weekend downloading 400GB of weights you can’t load.

Install path 1: Ollama (recommended for solo devs)

A distilled DeepSeek variant running under Ollama, exposed via HTTP – that’s what 95% of readers actually want.

# Linux one-liner
curl -fsSL https://ollama.com/install.sh | sh

# Start the daemon
ollama serve &

# Pull a variant that fits your GPU
# 8GB VRAM -> deepseek-r1:8b
# 16GB VRAM -> deepseek-r1:14b
# 24GB VRAM -> deepseek-r1:32b
ollama pull deepseek-r1:14b

# Smoke test
ollama run deepseek-r1:14b "Explain MLA in two sentences."

SitePoint’s 2026 deployment writeup pegs minimum practical specs for the Ollama path at 32GB system RAM, a 12GB+ VRAM GPU, and 400GB+ free NVMe if you plan to try larger quantized weights. Skip the NVMe and pulls will time out on the last shards.

Watch out: In your Modelfile, never set PARAMETER num_gpu -1 for anything approaching your VRAM ceiling. Full offload throws OOM at load time with no useful stack trace. Start with a conservative number (30-35 layers on 24GB) and raise it while watching nvidia-smi.

Install path 2: The real V3.2-Exp with SGLang or vLLM

If you actually have the GPUs, here’s the official launch line from the V3.2-Exp README:

python -m sglang.launch_server 
 --model deepseek-ai/DeepSeek-V3.2-Exp 
 --tp 8 --dp 8 
 --enable-dp-attention

vLLM has day-0 support for V3.2-Exp too. Both frameworks pull weights from Hugging Face automatically – no manual git-lfs dance. The --tp 8 --dp 8 flags assume an 8-way tensor-parallel plus data-parallel layout, the reference configuration. Adjust to your cluster.

Sparse attention kernels live in a separate repo called FlashMLA. If your framework version doesn’t bundle them, the server falls back to the dense attention path and loses most of the DSA speedup that made V3.2 worth deploying.

First-time config and verification

Once ollama serve (or your SGLang process) is up, hit the OpenAI-compatible endpoint directly:

curl http://localhost:11434/v1/chat/completions 
 -H "Content-Type: application/json" 
 -d '{
 "model": "deepseek-r1:14b",
 "messages": [{"role": "user", "content": "ping"}]
 }'

You want an HTTP 200 with a choices[0].message.content field. First call hangs for up to 20 seconds – that’s cold VRAM load, not a bug. Response comes back but throughput is 2 tokens/sec on a beefy GPU? Layers are spilling to CPU. Run ollama ps and check the PROCESSOR column; anything above 0% CPU means drop a quant level.

Common errors nobody warns you about

These are the ones that eat the most hours:

1. V3.2-Exp RoPE bug (pre-Nov 17, 2025): The official README notes an implementation discrepancy in the Rotary Position Embedding inside the indexer module of the inference demo. If you cloned before that date, output quality silently degrades – no crash, just worse answers. Re-pull the repo.
2. FP8 weights on pre-Hopper GPUs: The reference V3 weights ship in FP8 mixed precision. Consumer cards before RTX 40-series lack native FP8 tensor cores, so you either dequantize (large VRAM cost) or requantize to INT8/INT4. Most tutorials skip this entirely.
3. “Model loads but errors on init”: Community-reported on 32B+ quants running against 48GB VRAM. Root cause appears to be double-loading – the runtime allocates VRAM, then a second worker tries to load again. Setting worker count to 1 explicitly resolves it in most cases.
4. Ollama pull hangs at 99%: The last shard is often the largest. Community reports on GitHub issues consistently trace this to short-timeout HTTP clients on flaky connections. Restart the pull – Ollama resumes.

There’s an open question worth sitting with here: how many of these errors will still apply six months from now? Ollama’s update cadence is fast, SGLang’s FP8 support is improving, and DeepSeek ships new variants every quarter. This list reflects the install landscape as of early 2026 – some of it will age out.

Upgrading and cleaning up

Ollama makes this trivial:

# Upgrade the runtime
curl -fsSL https://ollama.com/install.sh | sh

# Swap model version
ollama rm deepseek-r1:14b
ollama pull deepseek-r1:32b

# Full uninstall (Linux)
sudo systemctl stop ollama
sudo rm /usr/local/bin/ollama
sudo rm -rf /usr/share/ollama
rm -rf ~/.ollama

For SGLang/vLLM deployments, upgrades usually mean bumping the framework pip package and re-pulling the model repo. The V3.1-Terminus comparison endpoint DeepSeek offered was taken down Oct 15, 2025, so if you were pinning against that for regression tests, you’re on your own now.

FAQ

Is DeepSeek-V3 actually the best open source AI model right now?

On math and coding benchmarks, the answer at the time of writing leans yes – V3-0324 drew strong benchmark reports at release, and V3.2-Speciale showed strong competition-math results per DeepSeek’s December 2025 release notes. “Best” for your use case still depends on what you can actually run, and the gap between frontier closed models and open weights shifts every few months.

Can I really run this on a single 24GB GPU?

Not the full 671B model. What you can run is a distilled variant (deepseek-r1:32b at Q4_K_M) or a heavily quantized V3 build via llama.cpp with Unsloth’s dynamic quants. Both give you the DeepSeek behavior signature – thinking traces, coding strength, Chinese-language quality – but with the accuracy trade-offs of quantization. If somebody claims 671B on 24GB, they’re either wrong or redefining “run” to mean “technically loads and produces tokens at glacial speed.”

Which version should I actually pull today?

Depends entirely on your hardware. V3.2 if you have a cluster and want the latest thinking-plus-tool-use behavior. On a single consumer GPU, pull R1-distill-14B or 32B via Ollama and move on.

Now go pull the model that matches your VRAM, run one prompt, and check ollama ps. If the PROCESSOR column shows any CPU percentage, drop one size and pull again before you write another line of client code.