LiteLLM Proxy v1.89.3 Setup Guide: LLM Gateway Install

The question I keep seeing in Reddit threads and GitHub discussions: which version of LiteLLM proxy should I actually pin to right now? After the March 2026 supply chain incident, that answer matters more than the install command itself. This guide walks through deploying the current stable LiteLLM proxy as your LLM proxy gateway – with the version trap, the Prisma failure mode, and the memory problem that production tutorials skip.

LiteLLM is an OpenAI-compatible AI gateway. It exposes one API – /chat/completions, /responses, /embeddings, /images, /audio, /batches, /rerank, /a2a, and /messages (GitHub README) – across roughly 100 providers. Deploy it once; every internal app calls one endpoint with one set of virtual keys instead of juggling Azure, Anthropic, and Bedrock SDKs in parallel.

Pick the right version before you install anything

At the time of writing, the current stable Docker tag is v1.89.3 (the main-stable alias), with v1.90.0-rc.1 available as a release candidate – verify the latest tag yourself at the GHCR registry before pinning, because LiteLLM ships weekly minor bumps. Stable tags go through a 12-hour load test before publishing (per the GitHub README), which is why they’re safer than main-latest for anything beyond a laptop experiment.

The version trap: on March 24, 2026, versions 1.82.7 and 1.82.8 were compromised and pulled from PyPI. They contained a credential harvester and a Kubernetes lateral-movement toolkit (Armosec analysis). The window was roughly 3 hours – 10:39 UTC to approximately 13:38 UTC – but PyPI download volumes for LiteLLM run around 3.4 million per day, so real installs happened. Safe versions: ≤ 1.82.6 or any patched release after 1.82.8. If your requirements.txt or Helm chart uses a floating version, fix that now.

The 3-hour window is worth sitting with for a moment. Most teams assume supply-chain attacks linger for days before detection. Three hours at 3.4M daily downloads still means thousands of compromised pulls. Pinning an exact version hash – not just a semver tag – is the only protection that actually works here.

Verify the Docker image signature with cosign before any deploy. The signing key is pinned in the repo’s releases page, so: cosign verify --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.3/cosign.pub ghcr.io/berriai/litellm:v1.89.3. This wouldn’t have caught the PyPI attack (different vector), but it does catch registry tampering.

System requirements

Four CPU cores and 8 GB RAM minimum for real workloads – that’s the official floor per the LiteLLM deploy docs. For Kubernetes, the pattern from the same docs is cpu: "1" and memory: "4Gi" per worker, one worker per pod.

The database row is the one people skip and then regret. No database URL means no virtual keys, no spend tracking, no admin UI – roughly 80% of why teams deploy this proxy in the first place.

Install with Docker

Use the litellm-database image. It bundles Prisma and the migration logic, so pointing it at a Postgres URL is the full setup. Pull the pinned stable tag, not latest:

docker pull ghcr.io/berriai/litellm-database:v1.89.3-stable

# Create your config
cat > litellm_config.yaml <<'EOF'
model_list:
 - model_name: gpt-4o
 litellm_params:
 model: azure/my_azure_deployment
 api_base: os.environ/AZURE_API_BASE
 api_key: os.environ/AZURE_API_KEY
 api_version: "2025-01-01-preview" # check Azure for current preview versions
general_settings:
 master_key: sk-1234
 database_url: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
EOF

docker run 
 -v $(pwd)/litellm_config.yaml:/app/config.yaml 
 -e AZURE_API_KEY=your_key 
 -e AZURE_API_BASE=https://your-resource.openai.azure.com/ 
 -p 4000:4000 
 ghcr.io/berriai/litellm-database:v1.89.3-stable 
 --config /app/config.yaml

Port 4000, per the Docker quick start docs. Supabase or Neon connection strings work in place of a local Postgres – same database_url field.

One Docker Compose trap: the docs confirm that .env, config.yaml, and prometheus.yml must exist as actual files before docker compose up. If any of them are missing, Docker creates empty directories in their place and Prometheus fails silently. Create the files first, even if empty.

The catch: pip install and Prisma

Turns out pip install 'litellm[proxy]' does not pull Prisma. Set DATABASE_URL and the proxy crashes on startup: ModuleNotFoundError: No module named 'prisma'. GitHub issue #12793 documents this – the fix is two extra steps that nobody includes in their tutorials:

python3 -m venv litellm-venv
source litellm-venv/bin/activate
pip install 'litellm[proxy]'

# Not pulled in automatically
pip install prisma psycopg2-binary

# Generate the Prisma client (path varies by Python version)
SCHEMA=$(find $VIRTUAL_ENV -name schema.prisma -path '*/litellm/*' | head -1)
prisma generate --schema "$SCHEMA"

litellm --version
litellm --config config.yaml --port 4000

If prisma generate still misbehaves, the uvx 'litellm[proxy]' --config ~/.config/litellm/config.yaml route handles the entrypoint binary path correctly on systems where the venv resolution trips up.

First-time configuration and verification

Minimum viable config: a model list, a master key, and optionally a database URL. The master key authenticates admin calls – rotate it and never commit it.

# Health check
curl http://0.0.0.0:4000/health/liveness

# Generate a virtual key with a rate limit
curl -X POST 'http://0.0.0.0:4000/key/generate' 
 -H 'Authorization: Bearer sk-1234' 
 -H 'Content-Type: application/json' 
 -d '{"rpm_limit": 60, "max_budget": 10}'

# Test a completion with the new key
curl -X POST 'http://0.0.0.0:4000/chat/completions' 
 -H 'Authorization: Bearer sk-...' 
 -H 'Content-Type: application/json' 
 -d '{"model": "gpt-4o", "messages": [{"role":"user","content":"ping"}]}'

Correct startup prints LiteLLM: Proxy initialized with Config, Set models: followed by the model aliases. Missing that line means the config didn’t load – check the volume mount path before debugging anything else.

Common errors and what they actually mean

Collected from GitHub issues. These repeat constantly; the fixes don’t appear in the quick-start.

• The Client hasn't been generated yet, you must run prisma generate – covered above. Re-run prisma generate against the exact schema path inside your venv. Running with a Supabase database, migrations complete fine but the proxy fails immediately after – same root cause, same fix.
• prisma:warn Prisma doesn't know which engines to download for the Linux distro "wolfi" – this appears on Chainguard/Wolfi base images and falls back to debian engines (GitHub issue #17093). It’s a warning, not the real failure. Look further down the log – the actual error is usually an OpenSSL 3.x version mismatch.
• Prometheus container exits immediately under Docker Compose – all three files (.env, config.yaml, prometheus.yml) must be present as files before docker compose up, per the Docker quick start docs. Docker auto-creates missing paths as empty directories; Prometheus then can’t read its config and dies.
• Memory keeps climbing for days – worker recycling, not autoscaling. See the next section.

The memory problem fix

RAM accumulates under sustained load and never releases until a restart. This isn’t a leak in the traditional sense – it’s object retention in the Python process. The official answer from the production docs is worker recycling via --max_requests_before_restart, which maps to Uvicorn’s limit_max_requests:

# In Docker / Kubernetes CMD
CMD ["--port", "4000", "--config", "./proxy_server_config.yaml", 
 "--num_workers", "1", "--max_requests_before_restart", "10000"]

One worker per pod, scale horizontally with more pods – not vertically with more workers per pod. The Horizontal Pod Autoscaler can’t reason about latency accurately when workers stack inside a single pod. The routing strategy (simple-shuffle by default as of v1.89.3; alternatives are least-busy, usage-based-routing, and latency-based-routing per the production best practices docs) only distributes load across model replicas, not across memory-bloated workers.

How much does this actually matter? A team hitting 10,000 requests per day with no restart cycle has reported monitoring alerts from RAM growth within 3-4 days. Set the flag before your first production deploy, not after you get paged at 2 AM.

Upgrading and removing

Swap the tag, recreate the container. Migrations run automatically on startup via litellm_proxy_extras. For pip: pip install --upgrade 'litellm[proxy]', then re-run prisma generate – the schema changes between minors.

Uninstall: docker rm -f the container and drop the Postgres volume if you don’t need spend history. For pip: pip uninstall litellm prisma and delete the generated client at $VIRTUAL_ENV/lib/python*/site-packages/prisma. And rotate any provider API keys that were stored in .env – especially if you ran 1.82.7 or 1.82.8 at any point.

One thing worth asking before you upgrade: is the new minor version in the tested window? Check the GHCR tag list and confirm the main-stable alias has moved. If you’re running a critical pipeline, a one-week lag behind the latest stable is a reasonable policy – not paranoia.

FAQ

Do I really need Postgres, or can I run LiteLLM without a database?

No database needed to start. Skip database_url and the proxy runs as a stateless router. You lose virtual key management, spend logs, and the admin UI – which is most of what makes it useful for a team rather than a personal experiment.

What’s the difference between the litellm and litellm-database Docker images?

The litellm-database image bundles Prisma plus migration logic – point it at a Postgres URL and you’re done. The plain litellm image is leaner but expects you to handle Prisma client generation yourself, which makes sense if you’re building a custom image on top of it. For a first deploy with no existing image pipeline, litellm-database with the main-stable tag cuts out roughly 30 minutes of Prisma debugging. Switch later if you need a custom build.

How do I confirm I’m not running a compromised version?

Run litellm --version or docker inspect on the image and compare against the GHCR tag list. The compromised versions are specifically 1.82.7 and 1.82.8 – anything outside that window is clean per the published Armosec and Trend Micro advisories. That said, “not 1.82.7 or 1.82.8” is not a complete security posture. Also verify the cosign signature (cosign verify --key .../cosign.pub ghcr.io/berriai/litellm:v1.89.3) and pin the exact version in your manifest rather than relying on a floating tag. Floating tags can silently pull a future bad release; the cosign check only works if you’re pulling by digest or an immutable tag. Treat version pinning and signature verification as two separate controls, not alternatives.

Next step: Pull ghcr.io/berriai/litellm-database:v1.89.3-stable, write your config.yaml with one provider, and run the health check above. Once /health/liveness returns 200, add --max_requests_before_restart 10000 to your CMD before anything goes near production.