Skip to content
AI Coding

Nvidia Native GPU Rust: Run Kernels Today [Guide]

Nvidia announces native GPU programming in Rust via cuda-oxide and cutile-rs. Skip the wrappers - start on cutile-rs, paste one vecadd, know the three setup traps.

6 min readBeginner

Your Rust inference stack is finally one language – until the hot kernel. Then you’re back in CUDA C++ or a Python DSL, fighting build scripts and silent races. That split just got a real answer: Nvidia announces native GPU programming in Rust, and the kernels themselves can be Rust now, not wrappers.

September 2026. Two open tracks under CUDA Rust. Tile is the one you can run this afternoon on the right box. SIMT is the escape hatch when tiles can’t express the control you need. Neither is shipping as production gospel.

I’ve been living in the mixed-language tax for years – cust bindings here, a .cu file there, hoping the launch config matched. The new path doesn’t erase that history overnight. It does kill the biggest lie: that the kernel body had to leave Rust.

Why wrappers and old Rust-CUDA paths still hurt

Host in Rust, kernel elsewhere: two toolchains, two mental models, bugs that only show under load. Community NVVM-style stacks got kernels compiling. You still juggled nightlies, extra crates, and thin safety around raw launches.

Same buffer as input and mutable output? Data race the type system never saw. Launch shape that doesn’t match indexing? Runtime failure – or silent corruption. That’s the tax.

Start with cutile-rs (NVIDIA’s default recommendation)

You describe work on a tile of data; the compiler maps threads and layout. No architecture-specific thread math in your source. That recommendation isn’t community folklore – it sits in NVIDIA’s September 2026 CUDA Rust blog.

cutile-rs wants stable Rust 1.89+, CUDA 13.3 recommended (verify the README if you’re on a odd arch), Linux, and compute capability 8.0+. No custom LLVM. No pinned nightly. On crates.io. Already showing up in Hugging Face Grout and mistral.rs as of that announcement.

Setup is short on purpose:

  1. Linux + driver/CUDA 13.x + sm_80+ GPU.
  2. cargo new vecadd_demo && cd vecadd_demo
  3. cargo add cutile
  4. Paste the kernel below into src/main.rs, then cargo run.

Ownership crosses the launch boundary. Partition a mutable tensor into exclusive chunks; shared inputs stay shared. Alias the same tensor as exclusive output and input – the borrow checker stops you before the GPU sees it.

A working tile kernel you can paste

Spirit of the official elementwise add, shaped for the cutile quick start. Crate APIs move – if this fails to compile, diff against current cutile docs rather than forcing the snippet.

use cutile::prelude::*;

#[cutile::module]
mod kernel {
 use cutile::core::*;

 #[cutile::entry()]
 fn add<const B: i32>(
 z: &mut Tensor<f32, { [B] }>,
 x: &Tensor<f32, { [-1] }>,
 y: &Tensor<f32, { [-1] }>,
 ) {
 let tx = load_tile_like(x, z);
 let ty = load_tile_like(y, z);
 z.store(tx + ty);
 }
}

fn main() -> Result<(), Error> {
 let device = Device::new(0)?;
 let stream = device.new_stream()?;
 let x = api::ones::<f32>(&[1024]);
 let y = api::ones::<f32>(&[1024]);
 let z = api::zeros::<f32>(&[1024]).partition([128]);
 let c: Vec<f32> = kernel::add(z, x, y)
 .first()
 .unpartition()
 .to_host_vec()
 .sync_on(&stream)?;
 let errors = c.iter().filter(|&&v| (v - 2.0).abs() > 1e-5).count();
 println!("{}", if errors == 0 { "PASSED" } else { "FAILED" });
 Ok(())
}

Partition size sets exclusivity and grid together. Body runs once per tile as a logical thread. No threadIdx spaghetti.

When tiles aren’t enough: cuda-oxide

Warp ops. Explicit shared memory. Clusters. Classic “one thread’s worth of code.” That’s cuda-oxide – early alpha, custom rustc codegen: #[kernel] via MIR → Pliron → LLVM IR → PTX. Host and device in one file with #[cuda_module].

Harder bar: Linux, CC 8.0+, CUDA toolkit (12.x+/13.0+ depending on which doc page you hit), clang with full headers, and a pinned nightly. Read the repo rust-toolchain.toml as of the day you install. Blog pins drift – copying nightly-2026-04-03 when the tree wants nightly-2026-08-28 breaks rustc-dev/codegen install.

cargo +nightly-YYYY-MM-DD install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxide
cargo oxide new vecadd_demo
cd vecadd_demo
cargo oxide doctor
cargo oxide run

doctor first. Missing clang resource dirs surface as cryptic bindgen errors about stddef.h. First cargo oxide run compiles the backend from scratch – long wall clock. People walk away thinking it hung. Later runs hit cache.

Safety tools: DisjointSlice for per-thread exclusive writes, optional #[launch_contract] so prepare/launch paths check block shape. Raw LaunchConfig stays unsafe on purpose. Shared memory on SIMT still needs unsafe; Tile hides threads and shared memory entirely – that’s the real split, not just syntax sugar.

Pro tip: Oxide build “stuck”? Leave it. One-time codegen tax. Run cargo oxide doctor before filing issues – wrong nightly pin and missing clang headers dominate the noise.

Gotchas tutorials skip

  • Platform wall: Supported matrix is Linux + sm_80+. Windows, macOS, pre-Ampere cards – no official path stated.
  • Maturity split: cutile already sits in outside projects. Oxide expects breakage and holes.
  • Perf reality: Early oxide GEMM write-ups landed near ~868 TFLOPS on B200, about 58% of cuBLAS – fine for alpha, not a silent cuBLAS swap. cutile materials show high fractions of peak on tuned elementwise/GEMM; numbers will shift with releases.
  • Interop: NVIDIA plans C++/Python CUDA bridges so a frontend choice doesn’t lock the ecosystem. Planned ≠ finished bridge for your next release freeze.

Is Tile enough for every kernel you’ll ship? Probably not – hence SIMT. How often you actually need that hatch once the tile compiler owns mapping is the part worth testing on your own ops, not debating in the abstract.

FAQ

Is this production-ready right now?

No. Oxide is alpha. cutile is ahead and used outside NVIDIA, still experimental on a critical path. APIs will move.

Do I need nightly Rust?

Only on cuda-oxide. cutile-rs targets stable 1.89+. Want native kernels without rustc component fights? Stay on Tile. Outgrow it, then accept the pinned nightly and rerun doctor whenever the pin changes.

How is this different from older Rust CUDA projects?

Older community stacks often aimed Rust at NVVM/PTX with separate kernel crates and thinner host safety. You still owned the race windows at launch. CUDA Rust is NVIDIA-backed, ships two first-class models (SIMT codegen backend vs Tile IR JIT), pushes ownership across the launch boundary, and keeps host+device in single-source patterns. Still NVIDIA GPUs only – not portable GPU Rust. Inside CUDA, the kernel language barrier dropped.

Linux box, sm_80+ GPU, cargo add cutile, run the tile add. Skim the cuda-oxide book only when Tile can’t express the op. File repo issues – that’s how this leaves alpha.

Related Tutorials