Deploy LiteLLM 1.100.1 as LLM Load Balancer

Most teams shop for an “LLM load balancer” and end up with a half-configured reverse proxy. What you actually need is a gateway that load-balances as a side effect of correct model groups – not a shiny strategy dropdown. LiteLLM 1.100.1 is that gateway: OpenAI-compatible front door, multi-deployment routing, spend hooks when you give it a database.

This is a deploy guide. You’ll pin 1.100.1 (as of the Sep 10, 2026 PyPI/GitHub release; 1.100.0 landed Sep 6, 2026), stand up the proxy, wire a minimum multi-deployment config, prove health, and dodge install traps generic write-ups skip. Project home: BerriAI/litellm and docs.litellm.ai.

System requirements before you touch Docker or pip

Package metadata on PyPI for 1.100.1 sets the floor at Python >=3.10, <3.15. Miss that and everything else gets weird.

CLI docs put load-test headroom around 1.5k+ RPS. Your real ceiling is still provider RPM/TPM – and whether Redis shares router state across replicas. Hardware is rarely the first bottleneck.

Official download sources (pin these)

Use PyPI litellm==1.100.1, image tags on ghcr.io/berriai/litellm / docker.litellm.ai/berriai/litellm (v1.100.1, latest, legacy main-stable), and the compose file at docs.litellm.ai/docker-compose.yml. After day one, pin a version tag; floating latest is a moving target.

Install LiteLLM 1.100.1 step by step

Two paths. Docker path = UI + Postgres. CLI path = config-only router without the admin UI.

Path A – Docker Compose

# Download compose, then start (customize credentials before prod)
curl -sSLO https://docs.litellm.ai/docker-compose.yml
# Edit: set LITELLM_MASTER_KEY and a long random LITELLM_SALT_KEY - then never rotate the salt
docker compose up -d

Or smoke one-liner: curl -sSL https://docs.litellm.ai/docker-compose.yml | docker compose -f - up -d. Gateway on http://localhost:4000; UI at /ui. Sample compose often ships master key sk-1234 – change it before anything real.

Path B – CLI with uv

uv tool install 'litellm[proxy]==1.100.1'
export OPENAI_API_KEY=sk-...
litellm --version # must print 1.100.1

Bare pip install 'litellm[proxy]' on an old interpreter is how people “upgrade” and still run 1.83.x-era builds. Check python --version (≥3.10) or stay on uv. Full trap notes live in the errors section below – don’t debug routing on the wrong wheel.

1. Docker + Compose V2, or Python 3.10+.
2. Pin 1.100.1 explicitly.
3. Set master key + salt before storing provider secrets in the UI.
4. Keep port 4000 off the public internet until TLS terminates in front.

Funny thing about the “load balancer” shopping list: once three deployments share one model_name, shuffle/least-busy stop feeling like products and start feeling like dials. If your mental model is still nginx-with-tokens, the YAML below will look too small. That’s the point.

First-time config: minimum viable LLM load balancer

Same model_name, multiple deployments. Load balancing falls out of that. Default strategy is simple-shuffle – see the proxy load balancing docs. Docs themselves mark usage-based-routing “(bad for perf)”; treat it as a special case, not a peer default.

# litellm_config.yaml
model_list:
 - model_name: gpt-lb
 litellm_params:
 model: openai/gpt-4o-mini
 api_key: os.environ/OPENAI_API_KEY
 rpm: 500
 - model_name: gpt-lb
 litellm_params:
 model: openai/gpt-4o-mini
 api_key: os.environ/OPENAI_API_KEY_2
 rpm: 500
 - model_name: gpt-lb
 litellm_params:
 model: azure/gpt-4o-mini
 api_base: os.environ/AZURE_API_BASE
 api_key: os.environ/AZURE_API_KEY
 api_version: "2024-02-01"
 rpm: 300

router_settings:
 routing_strategy: simple-shuffle
 num_retries: 2
 timeout: 30

general_settings:
 master_key: sk-change-me

Pro tip:rpm/tpm on deployments feed the router. Multi-replica proxies need redis_host / redis_port / redis_password under router_settings or each pod counts alone.

Config-only Docker (no UI DB):

docker run --rm -p 4000:4000 
 -v "$(pwd)/litellm_config.yaml:/app/config.yaml" 
 -e OPENAI_API_KEY -e OPENAI_API_KEY_2 -e AZURE_API_KEY -e AZURE_API_BASE 
 -e LITELLM_MASTER_KEY=sk-change-me 
 ghcr.io/berriai/litellm:v1.100.1 
 --config /app/config.yaml

CLI: litellm --config litellm_config.yaml --port 4000.

Verify the install actually works

curl -s http://127.0.0.1:4000/health/liveliness
# process up (no auth)

curl -s http://127.0.0.1:4000/health/readiness
# DB status; 503 if a configured DB is unreachable

curl -s http://127.0.0.1:4000/v1/chat/completions 
 -H "Authorization: Bearer sk-change-me" 
 -H "Content-Type: application/json" 
 -d '{"model":"gpt-lb","messages":[{"role":"user","content":"ping"}]}'

Hit chat five times. Watch logs/headers for deployment IDs that move – that’s shuffle proof, not a single sticky backend. UI path: Playground after Models + Endpoints.

Common install errors and fixes

The catch is these show up in troubleshooting notes and GitHub far more than in feature lists.

• Wrong version after pip: From 1.84.0 onward, LiteLLM needs Python ≥3.10. On older interpreters, bare pip can silently resolve to something like 1.83.9 with no hard error – you think you got current. Fix: 3.10+ or uv tool install 'litellm[proxy]==1.100.1', then litellm --version. (CLI quick start documents the floor and the uv path.)
• Budgets ignored: No database → global budget checks never fire; virtual keys fail with No connected db.; master key has no budget. Fix: official compose (Postgres) or cap spend at the provider. Don’t trust max_budget in a DB-less process.
• Salt key regret:LITELLM_SALT_KEY encrypts UI-stored provider keys. Change it later and old ciphertext will not decrypt. Fix: restore the original salt from backup – not a fresh random string.
• Docker → host vLLM:api_base: http://0.0.0.0:8000 can work under host pip and fail inside a container namespace (see community report on Hosted_vllm connectivity). Fix: host.docker.internal, host-gateway IP, or --network host on Linux.
• Proxy up before Postgres: readiness/DB errors right after compose. Fix: Postgres healthcheck + depends_on with healthy condition, or restart litellm once DB accepts connections.

If readiness is 503 with a DB configured, fix connectivity before chasing model keys – orchestrators will kill the pod either way.

Upgrade and uninstall

Docker: stop stack → point compose at v1.100.1 (or newer stable) → docker compose pull && docker compose up -d. Same salt. Same Postgres volume.

uv/venv (official upgrade path): stop proxy → optional pg_dump → bump litellm[proxy]==1.100.1 → prisma generate against schema under site-packages/litellm_proxy_extras → prisma migrate deploy (or the migrate flags the upgrade doc lists) → restart. Image builds hide most of that. Venvs don’t.

1. Uninstall CLI:uv tool uninstall litellm or delete the venv.
2. Uninstall Docker:docker compose down; add -v only when you mean to wipe spend/key data.
3. Remove local config.yaml and env files that still hold provider keys.

FAQ

Do I need Redis to load balance?

No – not for one proxy process. Only when multiple replicas must share RPM/TPM and router state: set redis_host, redis_port, redis_password under router_settings.

Docker UI path vs config-only: which should I run first?

Learning day: official compose + UI, click-test providers. GitOps prod: versioned config.yaml, pin ghcr.io/berriai/litellm:v1.100.1, keep Postgres if you need virtual keys and budgets that actually enforce. Config-only without DB is a valid OpenAI-compatible router – budgets just aren’t real there.

Why did my “latest” install feel stuck on 1.83.x features?

Wrong wheel, not wrong YAML. Confirm python --version and litellm --version before you touch routing – details under Wrong version after pip above.

Next: drop the three-deployment gpt-lb YAML on disk, pin v1.100.1, hit /health/liveliness, then fire five identical chat completions and watch the balancer move.