Deploy OpenVLA: Vision Language Action Model Setup Guide

The first question everyone asks about OpenVLA is the same: will this run on my 4090, or do I need to rent an H100? Short answer – yes, a 4090 works. Long answer takes a few paragraphs, because getting the vision language action model from git-clone to a running predict_action() call involves at least three dependency traps the README doesn’t warn you about clearly enough.

OpenVLA is an open-source 7B model that takes an RGB image plus a natural-language instruction and outputs a 7-DoF robot action. The flagship openvla-7b was built on top of the Prismatic prism-dinosiglip-224px VLM – fused DINOv2 + SigLIP vision backbone with a Llama-2 LLM – and trained on 970K trajectories from Open X-Embodiment. The code is MIT-licensed on GitHub; the weights inherit the Llama-2 community license. That distinction matters if you’re planning to ship a product.

Can your machine actually run it?

Straight from the OpenVLA paper (arXiv:2406.09246) and community deployment notes – as of mid-2024 when the paper dropped, these numbers have held up in practice:

The 8-bit row surprises most people. Section D.4 of the paper documents it: 8-bit drops to 1.2 Hz on the A5000, which is too slow for real-time control loops. 4-bit uses less than half the VRAM of bfloat16 and runs at 3 Hz with task performance the paper describes as comparable. If you’re memory-constrained, jump straight to 4-bit – do not stop at 8-bit.

There’s something worth sitting with here: a 7B model designed to control physical robots has to fit in real-time constraints that software models never face. 6 Hz sounds fine until you’re watching a robot arm drift because inference fell behind. The VRAM table above isn’t just benchmarking trivia – it’s the difference between a model that closes the loop and one that doesn’t.

Minimums I’d actually recommend: Ubuntu 22.04, CUDA 12.1+, one GPU with 16 GB+ VRAM, 32 GB system RAM, and roughly 30 GB of free disk for the checkpoint plus dependencies.

Install: the version that actually works

The README pins specific versions for a reason – the maintainers hit regressions in later releases of transformers, timm, and tokenizers. Don’t second-guess these pins on your first install. Get it running, then experiment.

# 1. Fresh conda env
conda create -n openvla python=3.10 -y
conda activate openvla

# 2. PyTorch 2.2.0 with the CUDA build matching your driver
pip install torch==2.2.0 torchvision==0.17.0 --index-url https://download.pytorch.org/whl/cu121

# 3. Clone and install OpenVLA (editable)
git clone https://github.com/openvla/openvla.git
cd openvla
pip install -e .

# 4. Flash-Attention 2 (training + fast inference)
pip install packaging ninja
ninja --version; echo $? # must print 0
pip install "flash-attn==2.5.5" --no-build-isolation

Inference-only and don’t care about training? Skip step 4. The official README includes a minimal requirements file: pip install -r https://raw.githubusercontent.com/openvla/openvla/main/requirements-min.txt. Then load the model with attn_implementation="sdpa" instead of flash_attention_2.

The three install traps nobody documents well

This is the section I wish had existed when I first tried this.

Trap 1: the wrong prismatic package. Turns out PyPI has a prismatic 0.4 package – a JSON serializer by Adam Byrtek, completely unrelated to OpenVLA. If a plain pip install prismatic runs anywhere in your workflow, you get the wrong one. GitHub issue #301 is full of people who hit this and got silent import failures at model load time. Diagnosis: pip show prismatic – if it says “Author: Adam Byrtek,” you’re broken. Fix: pip uninstall prismatic -y, then pip install -e . from inside the openvla directory. The editable install registers the local package correctly.

Trap 2: flash-attn 2.5.5 won’t build on modern CUDA. The build fails on CUDA 12.4+ with an undefined symbol error – the pinned version predates those drivers. The RoboVerse deployment docs recommend bumping to flash-attn 2.7.4 for stable CUDA 12 environments, even though the official README still pins 2.5.5. So: either hold at CUDA 12.1, or install 2.7.4 and accept you’re slightly off the tested path. Both work.

Trap 3: flash_attention_2 + missing device_map. The original inference snippet crashed on initialization for a lot of users. The updated README now includes device_map="cuda:0" with a comment calling it out explicitly. If you copy an older tutorial, add it manually – it’s one argument.

Pro tip: Before pip install -e ., run pip cache remove flash_attn. Stale cached wheels compiled against a different PyTorch version cause the most confusing runtime errors – they “install” fine, then throw undefined-symbol crashes on import.

First run: verifying it actually works

Skip robot hardware for the first test. Load the model, feed a dummy image, check that predict_action returns a 7-element numpy vector.

from transformers import AutoModelForVision2Seq, AutoProcessor
from PIL import Image
import numpy as np, torch

processor = AutoProcessor.from_pretrained("openvla/openvla-7b", trust_remote_code=True)
vla = AutoModelForVision2Seq.from_pretrained(
 "openvla/openvla-7b",
 attn_implementation="flash_attention_2", # or "sdpa" if no flash-attn
 torch_dtype=torch.bfloat16,
 low_cpu_mem_usage=True,
 trust_remote_code=True,
 device_map="cuda:0",
)

image = Image.fromarray(np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8))
prompt = "In: What action should the robot take to pick up the red block?nOut:"
inputs = processor(prompt, image).to("cuda:0", dtype=torch.bfloat16)
action = vla.predict_action(**inputs, unnorm_key="bridge_orig", do_sample=False)
print(action.shape, action) # expect (7,)

First load pulls roughly 14-15 GB from HuggingFace – plan for a wait on a home connection. If action.shape prints (7,) and VRAM peaks near 15 GB without the process dying, your bfloat16 setup is healthy.

What OFT and FAST changed

If 6 Hz feels slow for your application, here’s a fair question: what would fast enough look like? Two 2025 releases address that from different angles, and they’re not interchangeable.

OFT (released 2025-03-03) is the bigger deal for closed-loop control – 25-50x faster inference with continuous actions, according to the official README changelog. FAST (2025-01-16) takes a different angle: it compresses action chunks into fewer tokens, speeding up discrete-action inference up to 15x vs the default 256-bin discretization. One improves the model’s output representation; the other improves throughput on the same representation. If you’re doing real-time arm control, OFT is probably what you want. Running LIBERO simulations for a paper? The base checkpoint is fine.

Uninstall and clean slate

The model checkpoint is the big footprint. Cached weights live under ~/.cache/huggingface/hub/models--openvla--openvla-7b/. Full teardown:

• conda deactivate && conda env remove -n openvla
• rm -rf ~/.cache/huggingface/hub/models--openvla--openvla-7b
• rm -rf /path/to/openvla (the git clone directory)
• Optionally: pip cache purge to clear stale flash-attn wheels

Upgrading? A git pull inside the openvla directory picks up code changes because the install is editable. But if the pinned dependency versions in pyproject.toml have moved, recreate the env from scratch – mixing old and new pins is where subtle bugs live.

FAQ

Do I need a robot to try OpenVLA?

No. Feed synthetic images to predict_action – LIBERO simulation works fine. Robot hardware only matters when you close the control loop.

Why not just use vLLM to serve it?

You can, but it’s unofficial. OpenVLA isn’t on vLLM’s supported model list, and the fused SigLIP+DINOv2 visual encoder isn’t a standard vLLM component – behavior varies by vLLM version, and debugging a mismatch there is an unpleasant afternoon. For a single robot, the built-in REST server script already in the OpenVLA repo is more predictable and easier to validate. The vLLM path makes sense only if you genuinely need concurrent multi-robot inference and you’re willing to pin and test a specific vLLM version before trusting it with hardware.

What’s the difference between openvla-7b and openvla-v01-7b?

The v01 checkpoint is a pre-release artifact. Use openvla-7b.

Next step: Once predict_action returns a valid vector on random inputs, download 20 real demonstration episodes from BridgeData V2 and run them through the model. Compare predicted actions to ground-truth actions before you touch a physical robot. That’s the fastest way to catch normalization mistakes before they become expensive.