Deploy LiteLLM v1.101.0 LLM Gateway Open Source

End state first: LiteLLM v1.101.0 on http://localhost:4000, Postgres holding keys and spend, Admin UI up, one virtual key on the clipboard, and a live /v1/chat/completions call. Apps talk OpenAI format to a single endpoint. Deployment walkthrough for the current stable line – not a feature catalog.

LiteLLM (BerriAI) is an open-source proxy that maps 100+ providers into that interface, with budgets, load balancing, and logging when you wire a database. As of September 2026, pin v1.101.0 (release notes date it September 14, 2026: heuristic auto-router, semantic MCP tool search, off-peak pricing).

System requirements before you pull anything

As of September 2026, the proxy wants a modern runtime. Production guidance sizes 1 vCPU and 4Gi RAM per worker – set requests and limits to the same values. Their load tests sit on 4 CPU / 8 GB boxes. Undersize that and you get OOM kills and flapping probes.

Docker matches how most teams ship the gateway. CLI/uv is a laptop smoke test.

Official download sources for LiteLLM v1.101.0

Pin the tag. Rolling aliases move.

• GitHub: github.com/BerriAI/litellm
• Compose file: docs.litellm.ai Docker quickstart
• Container: ghcr.io/berriai/litellm:v1.101.0 (also docker.litellm.ai/berriai/litellm)
• PyPI: litellm==1.101.0 / litellm[proxy]

Cosign signatures land from commit 0112e53 onward (image security guide). Before prod:

cosign verify 
 --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub 
 ghcr.io/berriai/litellm:v1.101.0

main-stable still inches forward for old scripts but is deprecated. Prefer :latest (rolling stable) only if you accept drift – or pin v1.101.0.

Install the LLM gateway open source stack (Docker)

Full stack in one pull – gateway + Postgres – from the official compose file. Save it so secrets exist before first boot.

1. Keys (master must start with sk-):

   echo "sk-$(openssl rand -hex 32)" # LITELLM_MASTER_KEY
   openssl rand -hex 32 # LITELLM_SALT_KEY

2. Fetch, edit, start:

   curl -sSLO https://docs.litellm.ai/docker-compose.yml
   # Replace placeholder LITELLM_MASTER_KEY and LITELLM_SALT_KEY
   docker compose up -d

3. One-liner (placeholders only – throwaway):

   curl -sSL https://docs.litellm.ai/docker-compose.yml | docker compose -f - up -d

Compose brings Postgres 16 (POSTGRES_USER/PASSWORD/DB=litellm) and the proxy on host port 4000.

Think of LITELLM_SALT_KEY as a disk-encryption passphrase sealed into the volume: it encrypts provider credentials in the DB. Choose it once, before any model is saved. Official master-key rotation docs are clear – there is no supported in-place salt migration. Change salt later and every stored provider key is unreadble until you re-enter them. Back the salt up offline. Master rotation is documented and safe when a salt is already set; salt rotation is not.

Config-only single container (no DB)

Skip virtual keys, UI model CRUD, and spend tracking? Config file + one container:

cat > litellm_config.yaml <<'EOF'
model_list:
 - model_name: gpt-4o-mini
 litellm_params:
 model: openai/gpt-4o-mini
 api_key: os.environ/OPENAI_API_KEY
general_settings:
 master_key: os.environ/LITELLM_MASTER_KEY
EOF

docker run -d --name litellm 
 -v "$(pwd)/litellm_config.yaml:/app/config.yaml" 
 -e OPENAI_API_KEY="$OPENAI_API_KEY" 
 -e LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)" 
 -p 4000:4000 
 ghcr.io/berriai/litellm:v1.101.0 
 --config /app/config.yaml

No database → litellm_settings.max_budget never enforces (one startup warning, then silence). Virtual-key calls die with No connected db. Master key is the only credential, and it has no budget. Cap spend at the provider if you stay here. (Docker quickstart: “Running without a database”.)

CLI alternative (uv)

uv tool install 'litellm[proxy]'
export OPENAI_API_KEY=sk-...
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
litellm --config config.yaml

Python older than 3.10? Plain pip install 'litellm[proxy]' can quietly resolve to a pre-1.84 wheel instead of failing. uv tool install provisions a compatible interpreter – CLI quick start calls out 3.10+ for 1.84.0+.

First-time configuration after the containers are up

Open http://localhost:4000/ui. Sign in with the bootstrap credentials you set via env (master key is the admin surface until you create virtual keys).

Models + Endpoints → Add Model: provider, key (or os.environ/OPENAI_API_KEY), Test Connect, Add Model. Virtual Keys → create one → copy immediately; full token shows once.

Apps get the virtual key. Master key stays off laptops and CI logs.

Is a full gateway overkill for one script and one provider key? Sometimes yes. The moment a second team needs spend caps, a shared fallback, or “please stop pasting raw OpenAI keys into six repos,” the proxy stops being ceremony and starts being glue.

Verify the install actually works

Health docs split “process up” from “ready for traffic”:

# Process alive (no auth)
curl -s http://localhost:4000/health/liveliness
# Expect: I'm alive!

# Ready for traffic (includes DB; 503 if DB unreachable)
curl -s http://localhost:4000/health/readiness

# Completion through the gateway
curl http://localhost:4000/v1/chat/completions 
 -H "Authorization: Bearer sk-<your-virtual-key>" 
 -H "Content-Type: application/json" 
 -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Reply with exactly: gateway-ok"}]}'

Common install errors and fixes

LITELLM_MASTER_KEY not set – container exits. Long random sk-... in compose or -e. Sample placeholders are first-boot only; public scans still find sk-1234 on open gateways.

Address already in use: 0.0.0.0:4000 Map -p 4001:4000 or free the port (lsof -i :4000).

P1001: Can't reach database server at db:5432 Proxy beat Postgres, bad DATABASE_URL, or network isolation. Official compose uses depends_on: condition: service_healthy plus pg_isready. Check docker compose ps and logs; empty dev reset: docker compose down && docker compose up -d.

Error decrypting value, Did your master_key/salt key change recently? Volume still holds rows under an older salt/master. After throwaway experiments: docker compose down -v, then bring the stack up with the keys you intend to keep. Real data? Restore the original salt or re-register every credential – same class of failure as rotating salt after models exist.

YAML / mount errorsyamllint the file, mount at /app/config.yaml, don’t mount a directory onto a file path.

Upgrade, rollback, and uninstall

Read Breaking Changes on the release notes before you move. Support window (as of the mid-2026 policy post): roughly the four newest stable minor lines only.

docker pull ghcr.io/berriai/litellm:v1.101.0
# edit compose image tag →
docker compose up -d

Rollback: point the image at the previous stable tag (e.g. v1.100.x). Drop config keys the older binary rejects. Schema rollbacks may need manual migration care – use LiteLLM’s safe rollback notes when the DB already moved forward.

docker compose down # stop containers, keep DB volume
docker compose down -v # delete postgres_data (keys/spend gone)
docker rm -f litellm # single-container path
docker rmi ghcr.io/berriai/litellm:v1.101.0
# CLI: uv tool uninstall litellm

docker compose down alone leaves postgres_data. New salt/master on an old volume → decrypt errors until -v (or the original keys) comes back.

FAQ

Is LiteLLM free for production use?

Yes – the core proxy is open source and self-hostable. You pay compute, Postgres, and upstream provider bills. Vendor extras are optional; nothing above is required to run v1.101.0 on port 4000.

Docker or uv/CLI – which should I use?

Docker (or Kubernetes) when you want UI, virtual keys, and spend logs: compose already wires Postgres 16. On a laptop last Tuesday I only needed one model and no DB – so uv tool install 'litellm[proxy]', export two env vars, done. Same config shape either way; different process manager.

Why did budgets and virtual keys stop working after I dropped Postgres?

They never lived in the proxy process alone. Without a connected DB, virtual keys fail and global max budget does not enforce – you’ll get that one-time startup warning, then unlimited traffic until the provider cuts you off. Master key remains the only auth surface and carries no budget. If spend caps matter, keep Postgres; don’t expect config flags to police traffic the database was supposed to track.

Next: pin ghcr.io/berriai/litellm:v1.101.0, set real master + salt keys once, docker compose up -d, open /ui, mint one virtual key, point your OpenAI SDK base_url at http://localhost:4000.