CPU Inference LLM: Install llama.cpp b10435

Two paths for a CPU inference LLM stack on llama.cpp: compile for max ISA tuning, or pin a prebuilt/package and ship. Most boxes should take the second path. You dodge toolchain drift, lock release b10435 (as of 14 Aug 2026), and only open CMake when a binary dies with illegal instruction or you need a custom BLAS flag.

llama.cpp is the C/C++ engine under a large share of local GGUF runtimes; project home is llama.app. Below is a CPU-only deploy path – requirements, download, install, first run, verify, breakage, upgrade, cleanup – not another “what is local AI” tour.

System requirements for CPU inference

OS: 64-bit Linux (Ubuntu-class matches the official CPU tarballs), Windows 10/11 x64 or arm64, or macOS (Intel or Apple Silicon). C++17 tooling matters only for source builds – CMake 3.14+, Git, GCC/Clang/MSVC.

Those figures are not “file size equals RAM.” KV cache grows with context; leave headroom. AVX2-era x86 or modern ARM is the practical floor – older chips still run if you build for them, but prebuilts may not.

Pinning a build tag feels pedantic until the third silent upgrade breaks a script at 2 a.m. Treat b10435 like a dependency version, not a suggestion.

Download / official sources (b10435)

Take artifacts from the project only:

• Release binaries: github.com/ggml-org/llama.cpp/releases/tag/b10435 – CPU assets include llama-b10435-bin-ubuntu-x64.tar.gz, llama-b10435-bin-win-cpu-x64.zip, macOS arm64/x64 tarballs.
• Installer site: llama.app (Windows one-liner documented there).
• Package managers: Winget, Homebrew, conda-forge (install.md).
• Docker image names: ghcr.io/ggml-org/llama.cpp:server (also :light / :full in the Docker docs).

Skip random “llama-cpp.com” mirrors if supply chain matters. Build tags move fast – pin b10435 in scripts so CI does not float overnight.

Install llama.cpp for CPU (recommended paths)

Pick one. Package manager is fastest; prebuilt zip matches the tag; source is the escape hatch.

1) Package manager

# Windows
winget install llama.cpp
# or explicit id often listed as:
# winget install -e --id ggml.llamacpp

# macOS / Linux (Homebrew)
brew install llama.cpp

# conda-forge (Win/Mac/Linux)
conda install -c conda-forge llama.cpp

Per the official install doc, packages track new releases. Useful. Still verify the binary version after install if a brand-new model refuses to load.

2) Prebuilt b10435 (Linux x64 example)

curl -L -o llama-b10435.tar.gz 
 https://github.com/ggml-org/llama.cpp/releases/download/b10435/llama-b10435-bin-ubuntu-x64.tar.gz
mkdir -p ~/llama-b10435 && tar -xzf llama-b10435.tar.gz -C ~/llama-b10435
export PATH="$HOME/llama-b10435:$PATH" # adjust if binaries sit in a subfolder
# Windows: Expand-Archive the win-cpu-x64 zip and add that folder to PATH

3) Source CPU build

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout b10435 # pin
cmake -B build
cmake --build build --config Release -j$(nproc 2>/dev/null || sysctl -n hw.ncpu)
# binaries land in build/bin/ (llama-cli, llama-server, ...)

Docker alternative for a locked server:

docker run --rm -p 8080:8080 -v "$HOME/models:/models" 
 ghcr.io/ggml-org/llama.cpp:server 
 -m /models/your-model.Q4_K_M.gguf --host 0.0.0.0 --port 8080 --threads 8

First-time configuration (minimum viable CPU run)

README quick start once llama is on PATH (source builds may still expose hyphenated names):

# Pull a tiny GGUF and chat (downloads from Hugging Face)
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# OpenAI-compatible HTTP + web UI
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

# Source-build style
./build/bin/llama-cli -m ~/models/model.Q4_K_M.gguf -t 8 -c 4096 -n 128 -p "Say hello in one sentence."
./build/bin/llama-server -m ~/models/model.Q4_K_M.gguf --host 127.0.0.1 --port 8080 -t 8 -c 4096

The catch: set -t / --threads to physical performance cores – not every hyperthread, not every E-core. Decode is often memory-bandwidth bound; oversubscription adds contention and spin-wait noise. Tuning notes and a Texas A&M llama.cpp study (arXiv:2505.06461) both land on architecture-aware counts. Start at P-core count, run llama-bench, nudge ±2.

Context: start modest (-c 2048 or 4096). Raise it only after you watch RSS under load.

When is “CPU only” enough for you – private drafts on a commute machine, or a small internal bot that must not touch a GPU queue? That answer decides whether you stop at 0.8B smoke tests or pull a 7-8B Q4 next.

Verify the install works

1. llama cli --help or ./build/bin/llama-cli --help – flags, not “command not found.”
2. Tiny HF model command above should stream tokens without OOM.
3. Server up: curl -s http://127.0.0.1:8080/health or a short /v1/chat/completions POST if your build exposes OpenAI routes.
4. Optional: llama-bench -m model.gguf -t 8 for baseline tok/s.

Help works but load fails? Almost always the GGUF (wrong format, truncated download) or RAM – not PATH.

Common install errors and fixes

Illegal instruction (core dumped) / SIGILL – Binary targets a newer ISA (AVX-512/native) than this CPU. Rebuild on the host with plain cmake -B build, or force portable flags such as -DGGML_NATIVE=OFF and disable AVX-512 if needed. Do not drop a laptop-native binary onto a minimal cloud CPU.

error loading model / bad magic – Not GGUF (safetensors, incomplete download). Re-fetch a real .gguf or convert with project tools.

OOM / killed during load – Model + context exceed RAM. Smaller quant (Q4_K_M → IQ4_XS / Q3), cut -c, close browsers, swap last.

cmake: command not found / Makefile deprecated – Install CMake; old Makefile path is gone.

Slow despite high CPU % – Bandwidth wall. Lower threads toward physical P-cores; more workers past that wall often cut tok/s.

Latest tag misbehaves on a new model – No formal “stable” lane yet (maintainers in project discussion). If b10435 breaks your case, try the previous build tag until the regression clears – Qwen’s llama.cpp notes lean the same way.

Upgrade and uninstall

Upgrade

• Winget: winget upgrade llama.cpp (or reinstall the id you used).
• Brew: brew update && brew upgrade llama.cpp.
• Prebuilt: newer llama-bNNNNN-... archive, replace the directory, leave models/ alone.
• Source: git fetch --tags && git checkout b10435 (or newer), wipe build/, CMake Release again.
• Docker: pull fresh ghcr.io/ggml-org/llama.cpp:server and recreate with the same mounts.

GGUF weights usually survive engine jumps. Load fails after an upgrade? Bump the engine again or re-download a GGUF from current converters.

Uninstall / cleanup

# packages
winget uninstall llama.cpp
brew uninstall llama.cpp
conda remove llama.cpp

# prebuilt / source
rm -rf ~/llama-b10435 ~/llama.cpp
# remove PATH entries you added

# docker
docker rm -f llama-server 2>/dev/null
docker rmi ghcr.io/ggml-org/llama.cpp:server

# models (optional - large)
rm -rf ~/models/*.gguf ~/.cache/huggingface/hub # only if you want HF cache gone

FAQ

Do I need a GPU for a useful CPU inference LLM setup?

No. llama.cpp is CPU-capable by design. Plan ~16 GB free for a 7-8B Q4-class chat model; 1-3B fits nearer 8 GB.

Prebuilt b10435 or build from source – which should I pick?

Grab the Ubuntu/Windows/macOS CPU archive or Winget/Brew when you want a known tag in minutes. Build when the prebuilt SIGILLs, when you need OpenBLAS/MKL flags, or when GGML_NATIVE must match a server you control. Classic failure: CI baked on a desktop AVX-512 box, then the same binary crashes on a cheap VPS – rebuild inside the VPS and the GGUF loads.

Why is generation slower after I “max out” threads?

You probably oversubscribed past the P-core / bandwidth sweet spot (see the thread note under first-time config). One worker per logical thread looks busy in top and still loses tok/s. Re-bench after each change; chasing every new build tag helps less than a honest thread sweep on the machine you actually ship.

Next: install b10435 via Winget, Brew, or the Ubuntu CPU tarball, run llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF, lock -t to P-cores, then pull a 7-8B Q4_K_M for real work.