Install MedSAM2 Latest: Medical SAM Deploy Guide

Deploy the latest medical SAM (MedSAM2) with exact install commands, GPU specs, checkpoint download, verification, and real error fixes for 3D/video segmentation.

Riley Brooks2026-08-096 min readIntermediate

The #1 mistake with medical SAM installs? Grabbing a random PyTorch wheel and assuming any CUDA will do. I burned half a day on a silent fallback to CPU, then a cascade of version clashes, before the model even loaded. Reverse that: pin the exact stack the repo ships, download the right checkpoints first, then verify before you touch real volumes.

MedSAM2 is the medical SAM you actually deploy in 2025 for 3D CT/MRI/PET and clinical video (ultrasound, endoscopy). One middle-slice box. Full volume mask. Or frame-to-frame track. It fine-tunes SAM 2.1-tiny on a large medical set (paper scale: >455k 3D pairs and 76k video frames) so you skip training a specialist U-Net when interactive annotation or a solid baseline is the goal.

System Requirements for MedSAM2

Minimum viable stack for inference:

OS: Linux first (Windows/WSL or macOS may work; official docs do not treat them as primary)
Python 3.12
NVIDIA GPU + CUDA 12.4-compatible driver (main path is GPU; see FAQ for the lighter baseline)
Disk space for the clone plus checkpoints – MedSAM2_latest.pt is 156 MB; download.sh also pulls several modality variants, EfficientTAM, and sam2.1_hiera_tiny

Component	Pinned / Notes
Python	3.12 (conda env)
PyTorch	2.5.1 + torchvision 0.20.1 (cu124 index)
Install mode	pip install -e “.[dev]”
Key checkpoint	MedSAM2_latest.pt (recommended)

Recommended: recent RTX/A-class card and a CUDA toolkit that matches the cu124 wheel. Large 3D stacks and long videos need real VRAM headroom for the hierarchical encoder and memory bank – size the box to your volumes, not a generic “AI workstation” checklist.

Original MedSAM (2D-focused, Nature Communications 2024) used Python 3.10 + PyTorch 2.0. Separate env. Do not mix.

Official Download Source

Clone from the official MedSAM2 GitHub. Weights live on Hugging Face wanglab/MedSAM2. Training detail and the GPU-for-inference note are in arXiv:2504.03600. Project hub: medsam2.github.io.

No separate binary drop. You clone main, run their download script. Weights are research/education oriented (check the HF license note).

Step-by-Step Installation (Latest MedSAM2)

Clean env only – not your old SAM or MedSAM env.

conda create -n medsam2 python=3.12 -y
conda activate medsam2

# Exact CUDA 12.4 wheel (Linux)
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu124

git clone https://github.com/bowang-lab/MedSAM2.git
cd MedSAM2
pip install -e ".[dev]"

bash download.sh

download.sh creates checkpoints/ and pulls MedSAM2_2411.pt, MedSAM2_US_Heart.pt, MedSAM2_MRI_LiverLesion.pt, MedSAM2_CTLesion.pt, MedSAM2_latest.pt (default choice), plus EfficientTAM bits and sam2.1_hiera_tiny.pt. Needs wget or curl. HF rate-limit or a partial pull is how people end up with a missing latest weight and mysterious default failures.

Optional Gradio UI (app.py):

sudo apt-get update && sudo apt-get install -y ffmpeg
pip install gradio==3.38.0 numpy==1.26.3 ffmpeg-python moviepy

Pro tip: Right after the torch line, run python -c "import torch; print(torch.__version__, torch.cuda.is_available())". False? Stop. Fix driver/toolkit match before clone drama.

Ever notice how medical imaging stacks fail in the boring middle – not the paper abstract, the wheel pin? Same story here.

Classic 2D MedSAM path (older): conda python=3.10, PyTorch 2.0 from pytorch.org, clone bowang-lab/MedSAM, pip install -e ., then drop the Google Drive checkpoint into work_dir/MedSAM/medsam_vit_b.

First-Time Configuration

Almost no config file for basic runs. Point scripts at checkpoints/MedSAM2_latest.pt (many entry points already default there). 3D CT script: input image folder in, segmentation folder out. Box goes on the middle slice; memory attention carries the rest.

Training later? Edit YAML under sam2/configs/ (dataset paths, train_video_batch_size vs your VRAM) and make sure sam2.1_hiera_tiny.pt is present if a recipe expects it.

Verify the Install Works

Smoke test from the repo root:

python -c "import torch; from sam2.build_sam import build_sam2; print('import ok', torch.cuda.is_available())"
ls checkpoints/MedSAM2_latest.pt

Then a real call (demo layout or your prepped folder):

python medsam2_infer_3D_CT.py -i CT_DeepLesion/images -o CT_DeepLesion/segmentation
# or
python app.py

Masks on disk, no CUDA stack trace. Gradio should print a local URL. README Colab links remain the escape hatch when local drivers misbehave.

Watching a 3D volume light up slice-by-slice after one box is still the moment it clicks that the memory attention actually carries context – not just another 2D click-loop.

Common Install Errors and Fixes

torch.cuda.is_available() == False or later CUDA runtime errors
Wrong wheel or driver. Reinstall the exact cu124 pin above. nvidia-smi should show a driver that can host CUDA 12.x. Mixing conda cudatoolkit with the pip wheel is a classic silent foot-gun unless versions truly align.

CUDA OOM on volume/video
Shrink resolution if preprocess allows, fewer slices per pass, torch.cuda.empty_cache(), restart the process. Still tight? Use the Efficient MedSAM2 scripts (eff_medsam2_infer_...) shipped for lighter/FLARE-style baselines (README Updates: 2025.07.05).

download.sh fails or MedSAM2_latest.pt missing
No wget/curl, network block, or HF hiccup. Install curl/wget, retry, or pull one file:

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="wanglab/MedSAM2", filename="MedSAM2_latest.pt", local_dir="checkpoints")

Gradio / app.py dies on numpy or ffmpeg
Pin numpy==1.26.3 and install system ffmpeg exactly as above. Core inference can work while app.py still explodes – different dependency surface.

Import / sam2 path errors
Missed pip install -e ".[dev]" inside the clone, or wrong cwd. Activate medsam2 every session.

Upgrade and Uninstall

Upgrade: cd MedSAM2 && git pull && pip install -e ".[dev]", then re-run bash download.sh (or fetch only newer weights). Watch README Updates for baseline drops.

Cleanup:

conda deactivate
conda env remove -n medsam2
rm -rf MedSAM2
# optional: rm -rf ~/.cache/huggingface

No leftover editable install, no half-pinned torch in another env.

FAQ

Is MedSAM2 the same as the original MedSAM?

No. MedSAM (bowang-lab/MedSAM, Nature Communications 2024, v1.0.0) is the 2D ViT-B line. MedSAM2 is the SAM 2.1-tiny + memory path for volumes and video. New work → MedSAM2 unless you need the old 2D checkpoint ecosystem or a specific 3D Slicer Lite tie-in.

Can I run this on CPU only?

Default inference expects a GPU – that is how the authors shipped the main model. For constrained boxes, switch to the Efficient MedSAM2 train/infer scripts and the FLARE-oriented notes in the repo instead of forcing the full tiny+memory stack onto CPU.

What is the minimum I need for a first successful 3D run?

Pinned cu124 PyTorch, the medsam2 conda env, editable install from the clone, and MedSAM2_latest.pt present under checkpoints/. Prep a small CT folder the way the demo script expects (or use the linked Colab). One middle-slice box, run medsam2_infer_3D_CT.py, confirm masks land. That single volume is the real health check. Next stops if you want them: Slicer plugin path on the project page, or the modality-specific weights (CT lesion / US heart) from the same HF repo.

Clone, pin the torch wheel, run download.sh, verify on a demo volume today. First clean mask means you own the stack.