CogVideoX1.5 Diffusers Install: Official CLI Deploy

Two ways people actually get CogVideoX running: drop a ComfyUI workflow and hope the Manager sorts the nodes, or install the official Diffusers stack and drive it from the CLI/scripts. The first is friendlier for one-off art. The second is what you want for repeatable deployment, batch jobs, or anything you’ll later wrap in an API. This guide takes the Diffusers route for the current latest open weights – CogVideoX1.5-5B and its I2V twin – straight from the zai-org/CogVideo repo.

CogVideoX is the open T2V/I2V line from the Zhipu/Tsinghua side. As of the repo’s model table (1.5 series called out after the 8 Nov 2024 drop, still marked Latest on main), CogVideoX1.5-5B targets 1360×768, runs frames on a 16N+1 grid (default 81, about 5s at 16 fps, up toward ~10s with higher N), and is meant to run in BF16 on consumer cards when offloads are on. The architecture write-up – 3D causal VAE plus expert transformer – is in arXiv:2408.06072 (paper page updated Mar 2025).

System requirements before you touch pip

Don’t pull the weights only to find the card can’t finish a single denoise step.

Older CogVideoX-2B can sit near ~4 GB FP16 under the same diffusers tricks; plain 5B is listed around ~5 GB in the model table. The SAT weight path wants far more VRAM – tens of GB. Skip SAT unless you already know you need that stack.

Download the official source

Start at GitHub. Diffusers weight IDs on Hugging Face still often read THUDM/... even when the project org is zai-org – turns out both spellings show up across README links and cards, so a 404 is usually the wrong org prefix, not a missing model.

git clone https://github.com/zai-org/CogVideo.git
cd CogVideo
python -m venv .venv
# Linux/macOS
source .venv/bin/activate
# Windows
.venvScriptsactivate

Weights download on first from_pretrained. For offline boxes, grab the Diffusers folders from the HF links in the README (CogVideoX1.5-5B / 1.5-5B-I2V), including the THUDM/CogVideoX1.5-5B card.

Install step-by-step (latest main)

As of current main, requirements.txt pins a hot stack: diffusers>=0.35.2, torch>=2.8.0, transformers>=4.57.1, accelerate>=1.11.0, imageio-ffmpeg, and friends. That list is exactly where older CUDA 11.x boxes and stale global torch installs explode.

1. Activate the venv.
2. Install a CUDA-matched PyTorch before the requirements file if a bare pip would drag in CPU or wrong-CUDA wheels:

   pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
   # swap cu124 for the wheel family your driver actually supports

3. Then:

   pip install -r requirements.txt

4. If the CogVideoX pipeline bits are missing from your diffusers build:

   pip install -U git+https://github.com/huggingface/diffusers

Optional for the lower VRAM path later: pip install torchao (or the diffusers-torchao notes linked from the README).

Windows note: installs that feel “usable” almost always add Triton wheels plus SageAttention after the base stack. Official CLI still runs without them – it just crawls.

Ever notice how video-model READMEs read like they were written on one blessed Linux box with a clean CUDA 12 stack? That’s the gap this pin list opens. Budget time for the torch wheel fight before you blame the model.

First-time configuration that actually runs

No YAML. Minimum path is the shipped CLI demo plus the three memory calls the docs keep repeating.

From the repo root (venv active):

cd inference
python cli_demo.py 
 --prompt "A red fox trots through fresh snow under a pale winter sky, soft light, cinematic" 
 --model_path THUDM/CogVideoX1.5-5B 
 --generate_type t2v 
 --num_frames 81 
 --num_inference_steps 50 
 --guidance_scale 6.0 
 --dtype bfloat16 
 --fps 16 
 --output_path ../output_fox.mp4

Inside inference/cli_demo.py (or your fork) keep:

pipe.enable_sequential_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()

Frame rule:num_frames must be 16N+1 (N≤10 per the 1.5 table). Default 81 ≈ 5s @16 fps. Values like 49 or 60 are the silent quality killers – pipeline may run, motion looks broken.

I2V? --generate_type i2v, --image_or_video_path your.png, model ID ending in -I2V. Plain 1.5-5B sticks to the documented 1360×768 pair; flexible sizing is an I2V-side concern.

Prompts: English, long, concrete. Training leaned on optimized long captions; the repo even ships a helper that expands short ideas through a stronger LLM first. Two-word prompts usually look meh.

Verify the install works

Success checklist:

• cli_demo starts with no import errors
• HF download bars on first run only
• A playable output_*.mp4
• Peak VRAM in the ballpark of the official table when the three offload/tile calls are on

Smoke test: --num_inference_steps 20 and --num_frames 17 (still 16N+1). Confirm a file lands, then spend the full 81-frame job.

Common install errors and fixes

The catch is almost never “secret flags.” It’s pins, IDs, and offload choice.

• CUDA OOM on 10-16 GB cards – sequential CPU offload + VAE slicing/tiling is the path that approaches the ~10 GB BF16 mark; it’s slow on purpose. enable_model_cpu_offload() is snappier and the classic self-own on 12-16 GB cards without extra quant. INT8/torchao on text encoder / transformer / VAE is the next lever; else drop to older 5B/2B weights.
• ImportError / pipeline missing / version chaos – torch or diffusers older than the pins. Recreate the venv, CUDA torch first, requirements second, git diffusers last.
• from_pretrained 404 – try THUDM/CogVideoX1.5-5B and the zai-org ID on the live HF card. huggingface-cli login if the org gates files.
• Windows is correct but glacial – Triton + SageAttention, same route Comfy users document. Not required for correctness.
• Garbage motion / ignored prompt – short or non-English text. Expand and translate before touching samplers.

Watching the first full-length clip finish is oddly satisfying – and a good reminder that most of the “magic” is frame math plus patient offloading, not a hidden CLI switch.

Upgrade path and clean uninstall

cd CogVideo
git pull
source .venv/bin/activate # or Windows equivalent
pip install -r requirements.txt --upgrade
# re-run cli_demo; confirm model IDs if the README table moved

Project updates on the same README also point at CogKit (called out around Mar 2025) for newer fine-tune/inference tooling. Treat it as an add-on, not a swap for basic Diffusers CLI inference.

deactivate
rm -rf CogVideo
# optional HF cache wipe
rm -rf ~/.cache/huggingface/hub/models--THUDM--CogVideoX*
# Windows: %USERPROFILE%.cachehuggingfacehub...

Delete leftover MP4s and any venv you created outside the clone.

FAQ

Can I run CogVideoX1.5 on an 8 GB card?

Borderline. Official diffusers figures start near 10 GB BF16 with full offloads; INT8/torchao claims from ~7 GB. Expect crawl-speed sequential offload and possible failure. Use 2B or a cloud GPU if 8 GB is the hard ceiling.

Diffusers or ComfyUI-CogVideoXWrapper – which should I keep long-term?

Comfy when you want node graphs and community workflows the same afternoon. Diffusers/CLI when the job must be scripted – batch folders, CI, or a thin FastAPI/Gradio wrapper that shouldn’t drift every time a custom node updates. Common setup: Comfy to explore looks, Diffusers for the queue that has to finish overnight.

Why does my 81-frame video look worse than the gallery?

Usually the prompt, not the GPU. Gallery clips ride long, concrete English captions; short prompts under-specify motion and background, so the 5B model fills gaps with mush. Second check: you changed guidance_scale / steps far from the cli_demo defaults (6.0 / 50) or mixed dtypes. Match the demo first. Change one knob at a time. Frame count already correct at 81? Leave it – breaking 16N+1 to “save VRAM” with random lengths is how people invent new failure modes.

Clone the repo, make the venv, run the fox prompt with sequential offload, open the MP4. That’s the whole proof the stack is alive. Everything after is frames, prompts, and quant.