Deploy Tabby v0.32.0: Self-Hosted Coding AI Guide

Install Tabby v0.32.0 as self-hosted coding AI with Docker or binary. Specs, config, health check, CUDA gotchas, upgrade and cleanup steps.

Jamie Lin2026-09-207 min readIntermediate

Most “self-hosted coding AI” guides sell you a one-liner Docker command and call it production-ready. That is the fastest way to a restart loop. Tabby is one of the cleaner on-prem Copilot-style stacks – no external DBMS, OpenAPI surface, consumer GPUs – but the install path splits hard between CUDA Docker and CPU binaries. Treat it like infra, not a toy demo.

This guide deploys Tabby v0.32.0 (GitHub Latest on Releases as of this writing) as a self-hosted coding AI server: requirements, pinned install, first config, health check, failure modes, upgrade, cleanup.

System requirements before you pull anything

Tabby is self-contained (SQLite under the hood). That is a feature until the filesystem is wrong.

Resource	Minimum	Practical
OS	Linux x86_64, Windows x64, or Apple Silicon	Ubuntu-class server for shared team use
RAM	~8 GB host class (community self-host guides)	More if you index large monorepos
Disk	~10 GB free	Local SSD only – not NFS
GPU	Optional CPU mode (slow)	NVIDIA + NVIDIA Container Toolkit for Docker; ~2 GB VRAM class for StarCoder-1B; ~8 GB for CodeLlama-7B int8
Compute	–	int8 wants Compute Capability ≥ 7.0 (or 6.1); one GPU per Tabby process

~8 GB VRAM for CodeLlama-7B in default int8 CUDA mode – that figure comes straight from Tabby’s FAQ. Same page: multi-GPU is multiple processes with CUDA_VISIBLE_DEVICES / HIP_VISIBLE_DEVICES, not one fat container. Data directory stays on local disk; NFS locking and SQLite corrupt each other.

Funny how often “lightweight SQLite app” still dies on the storage story. If your home lab loves NFS home directories, this is where that habit bites.

Official download / image source

Pin sources. Skip random mirrors.

GitHub project:TabbyML/tabby
Release assets (binaries):v0.32.0 assets – tabby_x86_64-manylinux2014.zip (CPU), CUDA-tagged zips, Windows MSVC zips
Docker (docs default):registry.tabbyml.com/tabbyml/tabby – tag v0.32.0 to freeze; bare latest moves
macOS: Homebrew tap tabbyml/tabby/tabby
Models:TabbyML/registry-tabby (StarCoder-1B, Qwen2-1.5B-Instruct, CodeLlama, Qwen2.5-Coder, …)

v0.32.0 brought mistral/embedding API kind support, generic OAuth, multi-branch indexing. Nice once the box is up. Irrelevant if the container never stays healthy.

Install Tabby v0.32.0 (Docker on NVIDIA)

Install the NVIDIA Container Toolkit first. Without it, --gpus fails before Tabby starts.

# Pin the version; use local data dir on a real disk
mkdir -p "$HOME/.tabby"

docker run -d 
 --name tabby 
 --gpus all 
 -p 8080:8080 
 -v "$HOME/.tabby:/data" 
 registry.tabbyml.com/tabbyml/tabby:v0.32.0 
 serve 
 --model StarCoder-1B 
 --chat-model Qwen2-1.5B-Instruct 
 --device cuda

docker logs -f tabby

SELinux hosts: mount $HOME/.tabby:/data:Z. The official Docker guide calls this out; a lot of copy-paste compose files still omit it, then model writes fail in confusing ways.

Compose with the same pin:

services:
 tabby:
 restart: unless-stopped
 image: registry.tabbyml.com/tabbyml/tabby:v0.32.0
 command: serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cuda
 volumes:
 - "$HOME/.tabby:/data"
 ports:
 - "8080:8080"
 deploy:
 resources:
 reservations:
 devices:
 - driver: nvidia
 count: 1
 capabilities: [gpu]

First boot pulls models into the volume. Slow link? Go grab coffee. Killing the container mid-download is how you get half-written blobs and mystery restart loops.

CPU-only Linux – binary, not the GPU image

libcuda.so.1 missing. That is the smell. Release notes since v0.30.0 say the Docker image expects NVIDIA; issue #4306 still shows llama-server demanding CUDA libs on pure CPU hosts even when you pass --device cpu. Use the manylinux zip from the v0.32.0 release page.

# Adjust asset name to the v0.32.0 CPU zip on the release page
unzip tabby_x86_64-manylinux2014.zip
cd dist/* # layout varies slightly by asset
chmod +x tabby llama-server
./tabby serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cpu

Apple Silicon

brew install tabbyml/tabby/tabby
tabby serve --device metal --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct

Metal is fine for one developer machine (Apple install docs). Shared team load? Point those users at a CUDA/ROCm box instead of stretching a laptop.

Pro tip: Create $HOME/.tabby and chown it to your user before the first Docker run. Root-owned volume → classic Permission denied on models/TabbyML the day you switch to a binary or a non-root compose user (see discussions around volume ownership such as issue #1808-class reports).

First-time configuration (minimum viable)

Open http://localhost:8080. Create the admin account on first visit. Completions can serve after that.

Optional: Tabby does not auto-create config.toml. Need HTTP backends (Ollama, OpenAI-compatible) or overrides later? Drop ~/.tabby/config.toml yourself (Docker: /data/config.toml). Start from the config.toml docs.

LAN-exposed host? Set a strong web/JWT secret through your process environment and prefer tokens from the admin UI for IDE clients. Do not leave bare :8080 on a public IP without auth and TLS.

Add one git repo in the UI (Settings → Repository) before you index the whole org. Indexing is where disk and CPU spikes show up.

Verify the install works

Health first. UI second. Extensions last.

curl -sS http://localhost:8080/v1/health | jq .
# JSON with version / device / model fields - see Tabby health API

docker ps --filter name=tabby
# or: tabby --version # binary / Homebrew

/v1/health is the contract. Browser swagger at /swagger-ui/ is a convenience check only.

curl -X POST 'http://localhost:8080/v1/completions' 
 -H 'accept: application/json' 
 -H 'Content-Type: application/json' 
 -d '{"language":"python","segments":{"prefix":"def fib(n):n ","suffix":"n return fib(n - 1) + fib(n - 2)"}}'

Only after health + a completion response should you point VS Code / JetBrains / Vim at the server URL.

Common install errors and fixes

error while loading shared libraries: libcuda.so.1 – GPU image on a host without NVIDIA libs, or embedding subprocess ignoring CPU wishes. Fix: manylinux CPU binary, or a real GPU host with Container Toolkit. Stop restarting the same image.
failed to create shim ... libnvidia-ml.so.1 / cannot select nvidia device – Toolkit or driver mismatch. Fix toolkit; confirm nvidia-smi on the host.
Failed to fetch model organization / connection refused to registry raw GitHub – outbound network, proxy, air-gap. Pre-download models or mirror registry access.
Permission denied under /data/models – volume owned by root. sudo chown -R "$USER:$USER" "$HOME/.tabby" and restart.
Docker on Apple Silicon “works” then dies on CUDA libs – default distribution is x86+CUDA oriented. Homebrew + Metal instead.

Need noise? -e RUST_LOG=debug on Docker or RUST_LOG=debug tabby serve ... on binary.

Upgrade and uninstall

Upgrade (Docker): keep the volume so admin users, tokens, and indexes survive.

docker pull registry.tabbyml.com/tabbyml/tabby:v0.32.0
docker stop tabby && docker rm tabby
# re-run the same docker run / compose up with the new tag
docker compose pull && docker compose up -d # if you use compose

Skim release notes when jumping minors – model defaults and image CUDA baseline have moved before (CUDA 12 base notes around 0.30). Pin the tag you tested.

Uninstall / cleanup:

docker stop tabby; docker rm tabby
docker rmi registry.tabbyml.com/tabbyml/tabby:v0.32.0
# wipe local state only if you mean it:
rm -rf "$HOME/.tabby"
# Homebrew:
brew uninstall tabby

Binary installs: delete the extracted directory and any systemd unit you added.

FAQ

Is Docker always the right install for self-hosted coding AI with Tabby?

No. NVIDIA Linux: pin v0.32.0 on registry.tabbyml.com/tabbyml/tabby. CPU-only Linux: release binary. Apple Silicon: Homebrew + Metal. Forcing the CUDA image onto the wrong host is still the #1 self-inflicted outage.

How much VRAM do I actually need for StarCoder-1B vs a 7B model?

Two IDE users hit completions while chat is warm – that is when “it fit yesterday” stops fitting. Ballpark from FAQ + registry practice: 1B-class completion models sit near ~2 GB VRAM; CodeLlama-7B is called out around ~8 GB in int8 on CUDA. Chat model + KV cache needs headroom on top. Tight card? Shrink the completion model first. Inline lag annoys more than slower chat.

Can one Tabby container use two GPUs?

Not as a single process – official FAQ limit is one GPU per instance. Run a second container or binary, pin with CUDA_VISIBLE_DEVICES (or HIP on ROCm). Split teams across endpoints or put a reverse proxy in front when one card saturates. Do not expect a single docker run to span devices.

Next: pinned docker run (or CPU binary) → /v1/health → one admin user → index a repo you already know. Then invite people.