Deploy Phi-4 Locally: Install Guide [2025]

The #1 mistake with Microsoft’s small LLM Phi-4 isn’t a bad prompt. It’s grabbing the full 14B build on an 8GB card (or an old Ollama binary) and watching the runner die mid-load. Pick the variant and runtime for your box first. Install second.

Phi-4 is a 14B dense decoder-only Transformer from Microsoft Research – MIT license, 16K context, released December 12, 2024, trained Oct-Nov 2024 on ~9.8T tokens (model card). The lighter sibling is Phi-4-mini (~3.8B). This guide is a working local deploy of those two lines: copy-paste commands, verification, and the errors people actually hit.

Which Microsoft small LLM path fits your machine

Three practical routes. Not interchangeable.

Under ~12GB VRAM or tight RAM → start with phi4-mini / phi-4-mini. Comfortable 12GB+ and you want max STEM punch → phi4 / phi-4. Foundry Local mins sit around 8GB RAM / 3GB free disk (16GB RAM / 15GB disk recommended – Microsoft Learn, as of the get-started docs). For Ollama 14B Q4, plan 16GB+ system RAM so quant + KV cache aren’t fighting the OS.

Small models tempt a false economy: “it’s only 14B, ship the big file.” Hardware doesn’t care about parameter marketing. It cares whether weights, cache, and context fit at the same time. Choosing mini first isn’t cowardice – it’s how you get a reply before lunch.

Install Ollama and run Phi-4 (cross-platform)

Most installs finish in under ten minutes after the download.

1. Install the runtime

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows PowerShell
irm https://ollama.com/install.ps1 | iex

Or grab the installer from ollama.com/download. Confirm with ollama --version.

Ancient NVIDIA stacks (CUDA 11-era) are a known footgun – community reports show undefined-symbol crashes before the model ever answers. Update the driver first.

2. Pull and chat

# Full Phi-4 14B (official tag ~9.1GB Q4_K_M, 16K context)
ollama pull phi4
ollama run phi4

# Lighter mini when VRAM/RAM is tight
ollama pull phi4-mini
ollama run phi4-mini

First run fetches weights. Later starts are quick. Exit with /bye.

Pro tip: Don’t hand-roll a Modelfile from raw microsoft/phi-4 safetensors unless you own the GGUF conversion. Bad HF→Ollama paths blow up on rope_factors_long.weight shape mismatches (expected 64, got 0 or 48). Official phi4 / phi4-mini library tags dodge that.

3. Minimum viable “config”

Defaults work. Longer reasoning sessions only – pin context and sampling:

FROM phi4
PARAMETER num_ctx 16384
PARAMETER temperature 0.3

Then ollama create phi4-reason -f Modelfile and ollama run phi4-reason. Keep num_ctx ≤ 16K unless you’ve measured RAM headroom.

Foundry Local on Windows (and macOS)

Want Microsoft’s catalog, automatic hardware variants, and no CUDA toolkit babysitting? Foundry Local.

# Windows
winget install Microsoft.FoundryLocal

# macOS - match current tap docs
brew tap microsoft/foundrylocal
brew install foundrylocal
# or: brew install microsoft/foundrylocal/foundrylocal

foundry --version
foundry model list
foundry model run phi-4-mini
# or
foundry model run phi-4

It pulls the GPU/CPU/NPU build that matches the machine and opens an interactive session, plus an OpenAI-compatible endpoint. No Azure subscription for local use. First model list may fetch execution providers once – normal per the public docs.

Verify the install actually works

Downloaded ≠ working.

1. Presence: ollama list shows phi4 or phi4-mini. Foundry: foundry model list after a successful run.
2. Smoke prompt: short multi-step arithmetic or a tiny Python function. Weak/empty reply usually means wrong template, OOM thrash, or pure CPU crawl.
3. API check (Ollama):

curl http://localhost:11434/api/chat -d '{
 "model": "phi4",
 "messages": [{"role": "user", "content": "Reply with exactly: pong"}],
 "stream": false
}'

JSON with a short assistant message = good. Foundry: same idea against the local service once foundry service status is up.

transformers path (when you need the HF weights)

Official weights: microsoft/phi-4. From the card’s usage section:

import transformers

pipeline = transformers.pipeline(
 "text-generation",
 model="microsoft/phi-4",
 model_kwargs={"torch_dtype": "auto"},
 device_map="auto",
)
messages = [
 {"role": "system", "content": "You are a concise technical assistant."},
 {"role": "user", "content": "Write a Python function that merges two sorted lists."},
]
print(pipeline(messages, max_new_tokens=256)[0]["generated_text"][-1])

Chat markers on the card: <|im_start|> / <|im_sep|> / <|im_end|>. Full BF16 wants serious VRAM (~28-30GB class). Smoke-test on one GPU, then quantize or move to vLLM if you care about throughput.

Skimming arXiv:2412.08905 helps only for one decision: does the synthetic-data-heavy STEM bias match your workload? If your traffic is casual chat, mini may be enough anyway.

Common install errors and fixes

Ever notice how install guides stop at “it runs on my machine”? Here’s the ugly pile.

• rope_factors_long.weight wrong shape – bad HF→Ollama conversion. Delete the custom model; ollama pull phi4.
• missing tensor 'output.weight' or SWA asserts (GGML_ASSERT(n_swa > 0)) on mini – runner too old for the tag. Upgrade Ollama, pull again.
• CUDA undefined symbol / runner terminated – driver/runtime mismatch. Update NVIDIA; don’t expect current runners on ancient CUDA 11 hosts.
• CUDA OOM or multi-minute first tokens – 14B Q4 + long context overshot VRAM (practical working set often ~10-12GB once cache is live, not just the ~9.1GB download). Drop to phi4-mini, cut num_ctx, or accept offload slowness.
• Repetitive or truncated assistant turns – early tokenizer/chat-template residue (shared BOS/EOS/PAD quirks, extra assistant tokens). Official Ollama tags ship a fixed template; keep temperature modest for math.

Upgrade, migrate, uninstall

Ollama: upgrade the app, then ollama pull phi4. Remove with ollama rm phi4. Tag delete covers most cleanups; blobs live in the platform models directory if you need a full wipe.

Foundry Local:winget upgrade --id Microsoft.FoundryLocal on Windows; uninstall via winget when done. Clear cached models through Foundry’s model commands if disk matters.

HF cache: hub cache holds transformers downloads – huggingface-cli delete-cache or a manual wipe after experiments.

No special Phi-3→Phi-4 weight migration. Fresh pull. Fine-tunes stuck on early broken templates need a re-export against a fixed tokenizer/chat template.

FAQ

Do I need a GPU to run Phi-4?

No. CPU runs phi4-mini fine. 14B on CPU is single-digit tokens/sec unless cores and RAM are strong.

Ollama or Foundry Local – which should I keep long-term?

You already script agents against :11434 and hop across open models? Keep Ollama. On Windows with NPU/GPU packs and zero CUDA babysitting, Foundry Local is less friction – Microsoft’s aliases, ONNX-style hardware picks. A common split: Foundry for desktop demos, Ollama for tooling.

Why does my 8GB laptop choke on phi4 even though the file is “only” 9GB?

Download size ≠ runtime size. Q4 weights near 9.1GB still need KV cache, runtime overhead, and headroom. At 16K context the working set climbs past what an 8GB GPU holds without heavy CPU offload – and offload feels like a hang. Use phi4-mini, shorten context, or move to 12GB+. Community VRAM tables put practical 14B Q4 around 9-12GB once cache is live; full BF16 is a different planet (~28-30GB).

Next action: ollama pull phi4-mini (or foundry model run phi-4-mini), one multi-step coding prompt, then decide if you still want the full 14B weights.