CUDA in Rust: calling GPU code versus writing GPU code

"Can I use CUDA from Rust?" has two different answers, because it is really two questions. One is whether a Rust program can drive an NVIDIA GPU: allocate memory, copy data, load compiled kernels and launch them. The other is whether the kernel itself can be written in Rust and compiled for the GPU.

The first is a host-side question. The second is a compiler question. Mixing them up leads to wrong expectations about maturity, tooling and—most importantly—safety.

Path one: Rust on the host, kernels from elsewhere

Crates such as cudarc (opens in a new tab) wrap NVIDIA's libraries so that Rust code can create a context on a device, obtain a stream, allocate device buffers and copy slices to and from them. cudarc also exposes NVRTC, NVIDIA's runtime compiler, so a program can compile CUDA C++ kernel source to PTX at runtime and load it as a module.

In this arrangement, the kernel is still CUDA C++ (or PTX produced some other way). Rust is in charge of everything around it: lifetimes of buffers, error propagation, ordering of streams, and the code that decides when to launch.

Rust's type system genuinely helps here. A device buffer can be an owned value that frees its memory when dropped. A copy can return a Result instead of an error code you forget to check. Borrowing rules can stop the host from mutating a buffer while it is still borrowed elsewhere in Rust code.

The launch is the unsafe boundary

What Rust cannot see is the kernel. From Rust's point of view, a loaded kernel is a named entry point in a compiled module. The compiler does not know its parameter types, how many elements it will touch, or whether it checks its own bounds.

So launching is, correctly, an unsafe operation in libraries of this kind. The unsafe block is a promise from the programmer, not a feature switch. A sketch of the obligations, written as labelled pseudocode:

// pseudocode: obligations at an unsafe kernel launch
unsafe {
  launch(kernel "scale", grid, block, args: (out, inp, n))
}
// The caller promises:
//  - argument types and order match the kernel's signature exactly
//  - every buffer is large enough for every index the kernel may touch
//  - no other work writes these buffers concurrently on another stream
//  - the buffers stay alive until the kernel has finished
//  - the launch configuration is valid for this device

Some of these obligations can be taken over by the library. In current cudarc, for example, its safe device-buffer types record events when a launch reads or writes them, use those events to order work across streams, and wait for them before freeing memory on drop. cudarc LaunchArgs (opens in a new tab) For those buffer APIs, lifetime and ordering are managed for you.

What remains is what no host library can see: that the arguments match the kernel's signature, that every index the kernel touches is in bounds, and that the kernel only writes buffers you intended to be mutable. That is why cudarc's launch is still marked unsafe. No wrapper can prove a CUDA C++ kernel's internal indexing correct.

Path two: writing the kernel in Rust

The Rust CUDA project (opens in a new tab) takes the other route. It provides a compiler backend that turns Rust code into NVVM IR and then PTX, plus device-side crates for things like thread indices and shared memory. The kernel becomes a Rust function.

That changes what Rust can check inside the kernel: types, ownership within the function, and ordinary language rules. It does not make a GPU program automatically safe. Thousands of threads writing into one output buffer still need a scheme that guarantees each index is written by at most one thread, and that scheme lives in the algorithm, not the type system. Device code also runs without the standard library and with GPU-specific constraints on what operations are available.

The project describes itself as being in early development after a 2025 reboot, and warns users to expect bugs and safety issues. That is honest, and it should set your expectations: it is an interesting option to evaluate, not a drop-in replacement for the NVIDIA toolchain.

Choosing between them

A practical question decides most cases: where does your kernel code come from?

If you need existing, heavily optimised kernels—matrix libraries, attention implementations, vendor code—you will be calling them. Rust's value is a safer, clearer host program around a well-defined unsafe boundary.

If you are writing new, relatively self-contained kernels and want one language across host and device, compiling Rust for the GPU is attractive, provided you accept the maturity and toolchain trade-offs.

Other Rust GPU approaches draw these boundaries differently; this article describes only the two above. For both, the runtime constraints are the same as in C++: an NVIDIA GPU and driver must be present, launches are asynchronous, errors may surface later, and the GPU only helps if the work is parallel and the data movement is affordable. None of the Rust code in this article was compiled or run; it is pseudocode to show where the obligations sit.

Start with what happens when you launch a kernel, then revisit why GPUs are not always faster. Ownership and lifetimes of buffers are covered in the memory and data movement section.