Install promptfoo 0.122.0 for LLM Evaluation

Most teams waste more budget on untested prompts than on tokens. promptfoo 0.122.0 is the MIT-licensed CLI for local, versioned LLM eval runs. After agreeing to join OpenAI (announced March 2026), it stayed open source – still something you install and own on your machine.

This guide is install-only: runtime gate, copy-paste channels, minimum config, verify, the breakage patterns that hit first-time setups, then upgrade and wipe. Not a YAML assertions workshop.

Funny part? Teams debate rubric design for weeks and still trip on a Node minor. Tooling friction is rarely glamorous, but it decides whether eval ever becomes a habit.

System requirements before you touch npm

Minimums from the self-hosting specifications for the CLI client. The optional Docker UI server wants more RAM and disk.

Breaking gate in 0.122.0: Node.js 20 support is gone. Release notes require ≥22.22.0; installation docs recommend Node 24 LTS. If node --version still shows v20.x, upgrade before any package install.

nvm install 24
nvm use 24
node --version

Bare metal sometimes needs a C/C++ toolchain when native deps rebuild: build-essential (Debian/Ubuntu), xcode-select --install (macOS), or Visual Studio Build Tools (Windows).

Official download sources for promptfoo 0.122.0

As of August 2026, stick to:

• npm – [email protected] on npmjs.com/package/promptfoo
• GitHub – source/releases at github.com/promptfoo/promptfoo (tag 0.122.0)
• Homebrew – brew install promptfoo (formula stable listed at 0.122.0)
• PyPI – pip install promptfoo (thin wrapper; Node still required underneath)
• Docker UI – ghcr.io/promptfoo/promptfoo:latest (pin a digest/tag in prod when you can)

Skip random mirrors. The npm CLI is the product. Docker is optional shared history for the UI – not a prerequisite for local evals.

Install promptfoo step by step

Global npm after Node 24 is active – that’s the default path.

node --version # must read ≥22.22.0
npm install -g [email protected]
# rolling:
npm install -g promptfoo

No global install? Pin with npx:

npx [email protected] --version
npx promptfoo@latest --version

Other channels:

• Homebrew (macOS/Linux): brew install promptfoo
• Python-only CI gates: pip install promptfoo then run promptfoo. Under the hood it shells to npx promptfoo@latest; first run can fetch ~50MB. PyPI blurbs may still say Node 20+ – ignore that for 0.122.0 and provision Node 24 in the job image.
• App dependency: npm install promptfoo --save (same Node floor).

CI tip: Lock the runner to Node 24 and call npx [email protected]. Unpinned latest is how breaking majors sneak in on Monday morning.

First-time configuration (minimum viable)

Scaffold, don’t invent YAML on day one:

promptfoo init --example getting-started
cd getting-started
export OPENAI_API_KEY=sk-your-key-here
# PowerShell: $env:OPENAI_API_KEY="sk-your-key-here"

Empty interactive project: promptfoo init. Browser wizard: promptfoo eval setup.

Tiny config shape once you edit by hand:

# promptfooconfig.yaml
prompts:
 - 'Translate to {{language}}: {{input}}'
providers:
 - openai:chat:gpt-4o-mini
tests:
 - vars:
 language: French
 input: Hello world
 assert:
 - type: contains
 value: Bonjour

Provider IDs drift. Match what your account allows and what Getting Started shows today. No cloud key yet? Point at local Ollama after the binary itself runs.

Verify the install works

promptfoo --version
# expect 0.122.0 (or your pin)

cd getting-started
promptfoo eval
promptfoo view

eval prints a table and writes under ~/.promptfoo (Windows: %USERPROFILE%.promptfoo). view opens the local UI. Version OK but auth fails? Binary is fine – fix the key next.

Optional shared UI:

docker pull ghcr.io/promptfoo/promptfoo:latest
mkdir -p ./promptfoo_data
docker run -d --name promptfoo_container 
 -p 3000:3000 
 -v "$(pwd)/promptfoo_data:/home/promptfoo/.promptfoo" 
 -e OPENAI_API_KEY=sk-abc123 
 ghcr.io/promptfoo/promptfoo:latest
# http://localhost:3000

No volume on /home/promptfoo/.promptfoo → history dies on restart. Keep replicas at 1: SQLite plus in-memory jobs don’t share across pods (self-hosting docs call both limits out).

Ever notice how “five-minute install” guides skip the hour you lose to optional native bindings? That’s the real onboarding tax. Below is the short list that actually shows up in tickets.

Common install errors and fixes

Still on Node 20 after 0.122.0. Package may install; runtime then misbehaves. Move to Node 24, reinstall the global CLI, re-check promptfoo --version.

libsql binding missing – shape from the troubleshooting docs:

Error: Cannot find module '@libsql/darwin-arm64'
Require stack:
- /path/to/node_modules/libsql/index.js

Windows variant: @libsql/win32-x64-msvc. Fix: reinstall with optional deps (npm install -g promptfoo@latest or npm install --include=optional). npx cache junk: clear the promptfoo npx entry, then npx -y promptfoo@latest.

Native build failures. Install the OS toolchain above, then npm rebuild (or install with build-from-source when your platform needs it).

OPENAI_API_KEY is not set. Export it, or set apiKey on the provider. Using model-graded asserts (llm-rubric, similar, …) without OpenAI? Override the grader via defaultTest.options.provider.

OOM on large suites. Avoid --no-write. Prefer --no-table and --output results.jsonl. Raise heap when needed:

NODE_OPTIONS="--max-old-space-size=8192" promptfoo eval

Upgrade, migration, uninstall

node --version # ≥22.22.0 first
npm install -g [email protected]
brew update && brew upgrade promptfoo
docker pull ghcr.io/promptfoo/promptfoo:latest
# recreate container with the same volume

No dedicated config migrator ships with this release set – smoke a small suite with promptfoo eval. Dual npm + brew installs? Upgrade both or PATH keeps serving the older binary.

Uninstall (only the methods you used):

npm uninstall -g promptfoo
npm uninstall promptfoo # project local
brew uninstall promptfoo
which -a promptfoo # expect empty

Removing the package does not delete history. Full wipe (destructive):

rm -rf ~/.promptfoo
# also clear PROMPTFOO_CONFIG_DIR / PROMPTFOO_CACHE_PATH / PROMPTFOO_LOG_DIR if customized

Docker: docker rm -f promptfoo_container, delete the host data dir only if the DB should go too.

FAQ

Do I need Docker to run LLM evaluation with promptfoo?

No. CLI + promptfoo view is enough. Docker is only for a persistent shared UI/API.

Why did my upgrade from an older promptfoo break on Node 20?

You hit the 0.122.0 breaking change. Someone bumped the package on a laptop still pinned to Node 20 for another project; eval started throwing or behaving oddly. Install Node 24 LTS, confirm node --version, reinstall the CLI – then touch YAML again.

npm, brew, or pip – which should CI use?

Default: official Node 24 image + npx [email protected] (or a global npm install with the same pin). Homebrew fits Mac agents. pip is fine when policy only allows Python packages – remember it’s an npx launcher, so Node still has to exist on the runner, and wrapper text that still implies Node 20 is enough is stale relative to npm 0.122.0.

Terminal open? Switch to Node 24, run npm install -g [email protected] && promptfoo init --example getting-started, set one provider key, then promptfoo eval. When that works end-to-end, drop the same pinned command into CI.