Deploy CheXagent-2-3b: Chest X-ray AI Install

The #1 mistake with chest X-ray AI CheXagent is installing the newest transformers and expecting from_pretrained to just work. The Hub checkpoint ships custom code. As of the StanfordAIMI/CheXagent-2-3b model card we checked, deps still pin transformers==4.40.0. Skip the pin and you get import/API breakage – or you end up monkey-patching transformers.__version__ the way some agent wrappers do. Fix the environment first; then the model loads.

CheXagent (Stanford AIMI) is a ~3B CXR vision-language checkpoint: view ID, disease checks, sectioned findings, phrase grounding, tubes/fractures, and related tasks. Weights: Hub id above. Paper trail sits at arXiv:2401.12208 (v2, 18 Dec 2024). Repo disclaimer is blunt – research only, not clinical use.

Medical VLM installs often feel like flat-pack furniture with half the Allen keys missing. The pins on the card are the missing keys; everything else is optional polish.

System requirements before you touch pip

Open model_chexagent/chexagent.py and you will see why laptop-only setups stall: the wrapper sets device = "cuda" and loads bf16. No stock CPU/MPS path in that class.

Download the official sources

git clone https://github.com/Stanford-AIMI/CheXagent.git
cd CheXagent
python -m venv .venv
source .venv/bin/activate # Windows: .venvScriptsactivate
pip install -U pip

Code repo first; weights pull on first from_pretrained. Hub tree size for StanfordAIMI/CheXagent-2-3b is listed around 12.6GB. Optional: RadPhi-2 decoder and CXR vision encoders in the StanfordAIMI collection on the project page. CheXbench JSON lives under StanfordAIMI/chexbench – drop it at evaluation_chexbench/data.json only if you run their eval scripts.

Install dependencies (pin or regret it)

No polished requirements.txt matches the current card line-for-line. Install from the CheXagent-2-3b card (pins may change – re-read the card before you copy):

pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu124
# pick the CUDA wheel tag that matches your driver; card lists torch==2.7.1 / torchvision==0.22.1
pip install transformers==4.40.0 accelerate sentencepiece protobuf
pip install opencv-python albumentations Pillow matplotlib einops pyarrow
pip install gradio rich requests # demos

Sanity check: python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))".

Pro tip: Already stuck on a newer transformers globally? Isolated venv. Community stacks that temporarily set transformers.__version__ = "4.40.0" before load are working around the pin – not fixing it. Use a clean pin.

Will these pins still be right six months from now? Maybe not. The failure mode stays the same: custom modeling code lagging current transformers.

First-time run: API wrapper vs Gradio

Empty config is enough – the class already points at StanfordAIMI/CheXagent-2-3b with trust_remote_code=True and device_map="auto".

Scripted smoke test (repo root so imports resolve):

python demos/run_examples.py

Loads CheXagent(), exercises view classification, disease ID, findings, grounding, tubes, temporal pair, NER, and similar calls against public sample imagery.

Gradio (file on main is app_demo.py; some README lines still say app_demos.py):

python demos/app_demo.py
# TabbedInterface on 0.0.0.0:8888 - report generation + visual grounding

Raw HF path without repo helpers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

name = "StanfordAIMI/CheXagent-2-3b"
tok = AutoTokenizer.from_pretrained(name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(name, device_map="auto", trust_remote_code=True)
model = model.to(torch.bfloat16).eval()

paths = ["/absolute/path/to/cxr.png"] # real filesystem path
prompt = "What is the view of this chest X-ray? Options: (a) PA, (b) AP, (c) LATERAL"
query = tok.from_list_format([*[{'image': p} for p in paths], {'text': prompt}])
conv = [{"from": "system", "value": "You are a helpful assistant."},
 {"from": "human", "value": query}]
ids = tok.apply_chat_template(conv, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids.to("cuda"), do_sample=False, num_beams=1,
 max_new_tokens=512, use_cache=True)[0]
print(tok.decode(out[ids.size(1):-1]))

Verify the install actually matches their setup

Target the README findings-generation row, not vibes. After CheXbench JSON is in place:

python evaluation_chexbench/axis_3_text_generation/run_findings_generation.py

F1CheXbert ballpark for the CheXagent 3B row: Macro-14 44.9, Micro-14 58.0, Macro-5 55.3, Micro-5 62.5, Avg 55.2. Wild misses → wrong revision, dtype mess, or a transformers skew that still “loads.”

Faster check: one local PA/AP through view_classification should answer in option style, not traceback.

Common install errors and fixes

The catch is almost never “AI magic.” It is pins, device assumptions, and paths.

• trust_remote_code / custom module import errors after a transformers upgrade – Reinstall transformers==4.40.0 in a clean env. This checkpoint is not a “always latest” consumer.
• CUDA out of memory – Free the GPU; stay bf16/fp16; one image; max_new_tokens=512 as in demos. The 12.6GB figure is disk weights, not peak VRAM.
• device='cuda' on CPU-only hosts – Expected with the stock class. CUDA box, or edit device lines yourself (unsupported).
• Gradio temp path / permission errors – app_demo.py writes tmp...png under paths it creates; writable cwd matters. Read-only containers need a volume.
• URLs vs paths in from_list_format – Helpers may fetch HTTP; the tokenizer path wants files it can open. Save bytes first (same pattern Gradio uses).
• README script name drift – Current main: demos/run_examples.py, demos/app_demo.py.

Upgrade, migrate, uninstall

No tidy semver GitHub Releases labeled “CheXagent vX.Y.” You track git main for demos/eval and the HF revision on CheXagent-2-3b.

# refresh code
cd CheXagent && git pull

# force re-download weights if corrupt
huggingface-cli download StanfordAIMI/CheXagent-2-3b --force-download

# uninstall / cleanup
deactivate
rm -rf .venv
# optional: rm -rf ~/.cache/huggingface/hub/models--StanfordAIMI--CheXagent-2-3b
rm -rf CheXagent

Embedded inside an agent stack? Sync that project’s model_dir and tool wiring when the checkpoint revision moves – stale paths fail louder than bad prompts.

After install, the useful follow-ons are CheXbench axis scripts and the Gradio report/grounding tabs. Still research sandboxes only.

FAQ

Which CheXagent checkpoint should I deploy right now?

StanfordAIMI/CheXagent-2-3b. The repo CheXagent class and Gradio demo target it.

Can I run this without a NVIDIA GPU?

Not with the stock wrapper. Picture a cloud box without a GPU attached: import may succeed, then the first .to("cuda") path explodes. Official path assumes CUDA. A CPU port means forking device lines; interactive drafting will feel slow.

Is CheXagent approved for real clinical reporting?

No. Stanford marks repo and models research-only. The paper abstract’s reader-study figure – about 36% resident time saving when drafting from model text – is an efficiency signal, not clearance. Human radiologist still signs; follow your institution’s IR/compliance rules.

Next action: venv → pin transformers==4.40.0 → clone → python demos/run_examples.py once CUDA is green → open :8888 only after that smoke test passes.