Skip to content
ChatGPT & LLMs

How to Run a Nintendo in ~1GB of Qwen Weights

Show HN dropped a working Nintendo in ~1GB of Qwen weights. Here's what nesformer actually does, two ways to run it, and the gotchas tutorials skip.

5 min readBeginner

Can you actually play Super Mario inside a language model’s weights?

Yes – with caveats. The Show HN title is literal: A working Nintendo in ~1GB of Qwen weights. Drew Breunig’s nesformer (dbreunig/nesformer, ~30 Oct 2026) stuffs an NES 6502, 2 KB WRAM, and cartridge PRG-ROM into a stock Qwen3.5-style checkpoint. Nobody trained it. A compiler wrote the weights. Frames match a normal emulator – wall clock sits about 150-290× behind a 1983 console (as of the project README).

This is the hands-on path: short background, two run modes with honest tradeoffs, the CLI path that finishes, and traps most write-ups bury.

What just dropped (short version)

Each 128×128 Gated DeltaNet row can act as a real memory cell – gate pinned open, beta at 1, one-hot keys/queries. That trick comes from Qwenputer (Divyansh Agrawal); nesformer puts the 6502 + RAM + ROM across seven circuit layers and one all-zero attention layer. Turns out the same “fast weights” memory shows up in Schlag, Irie, and Schmidhuber’s delta-rule work (arXiv:2102.11174).

Disk math first: 9.3M nonzero values zip to 3.3 MB. Expand them and you get 466.6M bf16 params (933 MB) in ordinary Qwen3_5ForCausalLM.from_pretrained layout – hidden 768, vocab 263808. The net only streams the CPU log. Host code paints the PPU, stuffs controller bits, and fires NMI at vblank. No APU. Mid-scanline PPU tricks stay approximate on purpose.

Think of it like burning a CPU schematic into ROM masks instead of letting silicon learn the chip – only the “masks” are transformer weights and the “clock” is greedy decoding.

Method A vs Method B: how should you run it?

Two engines. Pick on purpose.

Path Engine flag Rough speed (README, late Oct 2026) Deps Best for
A – Stock transformers --engine hf ~10 tokens/s torch ≥2.6, transformers ≥5.22 Proving the checkpoint is a normal HF model
B – Numba fast path --engine fast (default) ~1,650 tokens/s on one CPU core numpy, scipy, numba, pillow Frames and GIFs you’ll wait for

Real-time NES would want ~700,000 tok/s. Numpy path? ~210. Method A is a 3-frame sanity check. Method B is the one that finishes 600 frames (~10 s of NES time) in roughly 25 minutes after a one-shot ~20 s numba compile – ~165× the HF rate on the published numbers. Tutorial winner: B.

Detailed walkthrough: get Thwaite (then Mario) running

Python 3.11+. Pull the repo and nesformer-weights.zip from the GitHub release layout. Unzip so weights/ sits beside the nesformer/ package – not inside it.

  1. Venv + fast stack: 
    python3 -m venv .venv
source .venv/bin/activate # Windows: .venvScriptsactivate
pip install numpy scipy numba pillow
  2. From the top folder: shasum -a 256 -c weights/SHA256SUMS.
  3. Bundled free ROM first (no dump): 
    python -m nesformer run roms/thwaite.nes --frames 120 --show

    --show wants a 24-bit color terminal (iTerm2, Windows Terminal, …). First run compiles numba (~20 s). Then 1-6 s per frame is normal.

  4. Optional integrity pass: 
    python -m nesformer check roms/thwaite.nes --tokens 50000

    Stops on first mismatch. trace dumps tokens; info describes the checkpoint; build rebuilds weights in ~10 s.

  5. If you own a legal NROM dump (mapper 0), script inputs with absolute frames from power-on at 60 fps: 
    python -m nesformer run roms/smb.nes --frames 600 --show 
 --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375"

    Miss the title-screen window and Start never registers. Longer A ranges jump higher in SMB. Output defaults to a GIF; override with --gif PATH or --png DIR.

Same checkpoint, any NROM cart: host loads PRG into DeltaNet state at startup. No rebuild per game.

Edge cases that bite beginners

  • Mapper wall. iNES mapper 0 only (NROM – SMB, Donkey Kong, Excitebike). Anything else aborts. PRG is a fixed 32 KB table in the weights; bank switches aren’t wired (that would need full ROM + bank regs in state plus an address stage).
  • HALT. Unofficial 6502 opcodes or a jump into RAM stop the run. Code path must stay in cartridge PRG-ROM. BRK + NMI work; APU frame / DMC IRQs don’t. Reads in $4020-$7FFF return 0 (no PRG-RAM).
  • Button timing. Ranges are absolute frame indexes, not “when the menu appears.” Combine buttons with +. Fussy title screens: nudge Start a few frames earlier.
  • Engine surprise. Missing numba/numpy → pain. HF needs torch/transformers and crawls at ~10 tok/s – learn on it, don’t “play” on it.
  • Idle skip. Wait-for-vblank loops drop out (2.6-4.2× fewer tokens). Visible frames and I/O timestamps still match the reference. --no-skip only when you’re debugging timing itself.

Pro tip: Thwaite + SHA256 + a short check before any 600-frame GIF. Half an hour is a long time to learn the zip was corrupt.

PPU is a host scanline renderer. Sprite-0 and mid-scanline edge cases are approximate; silence is expected, not a broken install.

Why this matters beyond the meme

You watched a deterministic machine live in compiled transformer weights – nestest 3486 golden match, Thwaite 1,916,527 tokens / 600 frames clean, Full Game 2,645,885 / 420 clean, stock HF bf16 zero mismatches on 2,651 positions (README validation). Nearby threads worth opening: Qwenputer’s RISC-V-in-weights path, christopherpow nestest, linear RNNs as addressable RAM.

But here’s the part that sticks: the open question isn’t “will this replace Mesen?” It won’t. It’s how far compiled circuits inside foundation-model skeletons go before someone needs gradients again.

FAQ

Do I need a GPU?

No. Default fast path is numba on CPU. Torch will use a GPU if present on the HF engine – still the slow option here.

Is ten seconds of Mario really ~25 minutes?

On the README machine, yes. ~1,650 tok/s after compile, 1-6 s/frame, 600 frames ≈ 10 s NES time ≈ 25 minutes wall clock, idle skip already on. Lab demo, not couch play. Want the aesthetic only? --frames 60, then scale.

Can I rebuild the weights or use my own ROM?

python -m nesformer build regenerates the checkpoint in ~10 s. Legal NROM only – see edges above for mappers and HALT. Repo ships free test ROMs; bring your own dumps.

Clone, verify weights/SHA256SUMS, run 120 frames of Thwaite with --show. When the first frame paints in the terminal, the Show HN stops being abstract.

Related Tutorials