Writing GPU shaders in Rust
Firestar99
Gulaschprogrammiernacht 24 (GPN24) · Day 2 · ZKM Vortragssaal
Overview
In this GPN24 talk, "Writing GPU shaders in Rust," Firestar99, a former graphics programmer turned compiler maintainer, delves into the ambitious Rust GPU project. The core premise is to enable developers to write GPU shaders directly in Rust, addressing the pervasive frustrations associated with traditional shader languages like GLSL, WGSL, or domain-specific variants. The talk highlights the significant benefits of leveraging Rust's robust type system, tooling, and shared data structures across both CPU and GPU codebases, aiming to eliminate a common source of bugs: API mismatches between host and device.

Key moments
- 0:00 Introduction and motivation for Rust GPU
- 4:00 Rust GPU: Compiling Rust to SPIR-V and WGSL
- 8:00 Eliminating API Mismatches with Rustc Layouts
- 18:00 Understanding SPIR-V's Logical Pointer Limitations
- 22:00 Quasi-Pointer Emulation (QPO) for Rust Pointers
- 31:00 Tiny Wasm: Stress testing GPU drivers with 90MB shaders
- 34:30 Q&A: Rust GPU Community Fork and Future Outlook
- 43:00 Q&A: Comparing Rust GPU with NVIDIA's CUDA Oxide
Writing GPU shaders in Rust
Speakers: Firestar99
Conference: GPN24
YouTube: https://www.youtube.com/watch?v=I8llKgdYbYs
Overview
In this GPN24 talk, "Writing GPU shaders in Rust," Firestar99, a former graphics programmer turned compiler maintainer, delves into the ambitious Rust GPU project. The core premise is to enable developers to write GPU shaders directly in Rust, addressing the pervasive frustrations associated with traditional shader languages like GLSL, WGSL, or domain-specific variants. The talk highlights the significant benefits of leveraging Rust's robust type system, tooling, and shared data structures across both CPU and GPU codebases, aiming to eliminate a common source of bugs: API mismatches between host and device.
Firestar99, who transitioned from a user of Rust GPU for his master's thesis to a core maintainer and now works commercially on the project, provides a deep dive into the technical challenges and innovative solutions employed to make Rust a viable shader language. The presentation is not merely a theoretical discussion but a pragmatic exploration of compiler internals, intermediate representations, and the fundamental limitations of GPU architectures. It underscores the project's commitment to delivering a cohesive development experience for graphics and compute programming, pushing the boundaries of what's possible with Rust in performance-critical environments.
The talk is particularly relevant for graphics programmers, Rustaceans interested in low-level systems, and anyone frustrated by the current state of GPU programming. By demonstrating how Rust GPU tackles issues from memory layouts to pointer semantics, Firestar99 illustrates a path towards more reliable and maintainable GPU code, ultimately fostering a more integrated and less error-prone development workflow.
Background
▶ Watch: Introduction and motivation for Rust GPU (0:00)
Traditionally, writing programs for Graphics Processing Units (GPUs) involves specialized shader languages. The most prominent examples include GLSL (OpenGL Shading Language) and WGSL (WebGPU Shading Language). These languages, often resembling C in syntax but lacking many general-purpose programming features like pointers, are compiled into Intermediate Representations (IRs) such as SPIR-V. These IRs are then passed to graphics APIs like Vulkan or WebGPU at runtime for execution on the hardware. While functional, this bifurcated approach introduces significant friction: developers must manage two distinct codebases (CPU in Rust, GPU in a shader language), often leading to tedious manual synchronization of data structures and constant vigilance against API mismatches.
A prime example of this friction lies in data structure layouts. When sharing data between CPU and GPU, ensuring identical memory layouts is crucial. In Rust, the #[repr(C)] attribute helps achieve C-compatible layouts, but translating this to shader languages is far from trivial. WGSL, for instance, adheres to the std430 layout (or std140), which has specific and often counter-intuitive alignment rules. A vec3 in WGSL, for example, despite conceptually being three floats, has a size of 12 bytes but an alignment of 16 bytes. This violates fundamental layout rules typically enforced in C or Rust (where size must be a multiple of alignment), necessitating workarounds like using vec4 and wasting space, or manually packing fields. This disparity frequently leads to subtle, hard-to-debug errors when data is passed between the CPU and GPU.
The Rust GPU project emerged from this frustration, aiming to unify the development experience by allowing Rust to serve as the single language for both host and device code. The vision is to take Rust code as input, compile it into SPIR-V, and then allow that SPIR-V to be used with any compatible graphics API, including transpilation to WGSL for WebGPU targets. This approach promises to eliminate the constant struggle with language boundaries and API mismatches, leveraging Rust's powerful compiler and type system to ensure correctness from the outset.
Key Findings
▶ Watch: Eliminating API Mismatches with Rustc Layouts (8:00)
The Rust GPU project has made significant strides in enabling Rust as a first-class shader language, delivering several key findings and contributions:
- Unified Language for CPU and GPU: Rust GPU effectively compiles Rust code into SPIR-V, allowing developers to write both their CPU application logic and GPU shaders in a single, consistent language. This eliminates the need for context switching between Rust and traditional shader languages like GLSL or WGSL, streamlining development workflows.
- Elimination of API Mismatches: By using Rust for both sides, shared data structures can be defined once in a common crate and used directly by both the main application and the shader code. This inherently resolves the prevalent issue of API mismatches, where differing interpretations of data layouts between CPU and GPU languages lead to runtime errors and debugging nightmares. The project leverages
#[repr(C)]to ensure consistent memory layouts, avoiding the complexities ofstd430orstd140layouts found in other shader ecosystems. - Custom Compiler Backend and Toolchain Integration: Rust GPU operates as a custom compiler backend for
rustc. Thecargo gpu installtool orchestrates a complex build process, downloading specific nightly Rust versions and the Rust GPU backend, then compiling the shader crate twice: once for the CPU and once for the SPIR-V target. This sophisticated integration demonstrates the adaptability of the Rust compiler infrastructure. - Innovative Solutions for SPIR-V Limitations: SPIR-V, particularly for Vulkan shaders, presents unique challenges such as the absence of a standard library, allocators, recursion, and crucially, the use of logical pointers that do not support pointer arithmetic or references as function arguments. Rust GPU addresses these through:
- No-
stdEnvironment: Leveraging Rust's existingno_stdcapabilities, similar to embedded development. - Post-Link Legalization: A custom SPIR-V linker that performs transformations (like inlining functions) after initial codegen to resolve illegal SPIR-V patterns created by Rust's pointer semantics. This relies on a "closed world assumption" for shaders.
- Quasi-Pointer Emulation (qpointer): A sophisticated technique that emulates pointer semantics using pointer tagging (combining a tag and an offset) and control flow analysis. This allows for features like basic
allocfunctionality on the GPU, albeit with limitations.
- Benchmarking and Practical Limitations: While powerful, pushing Rust's full feature set to GPUs reveals current practical limitations. Compiling a complex project like Tiny WASM (a WASM interpreter) resulted in a 90MB SPIR-V binary, orders of magnitude larger than typical shader binaries (kilobytes). This extreme size often leads to driver crashes, highlighting that GPU drivers and hardware are optimized under the assumption that shaders are "small" and extensively inlined. This finding underscores the need for continued research into efficient compilation strategies for general-purpose GPU programming.
- Targeting Vulkan Shaders: The project specifically targets Vulkan shaders, which support both graphics and compute operations and are widely supported across devices, enabling transpilation to WGSL for web targets. This distinguishes it from other efforts (like LLVM's SPIR-V target or Zig) that often focus on OpenCL kernels, which are compute-only and rely on different pointer models.
Technical Deep Dive
▶ Watch: Quasi-Pointer Emulation (QPO) for Rust Pointers (22:00)
The technical foundation of Rust GPU lies in its ability to translate Rust's high-level abstractions into the low-level, constrained world of GPU execution environments. This involves tackling several architectural and semantic disparities between a general-purpose language like Rust and the specific requirements of shader compilation.
At an architectural level, a Rust GPU project typically consists of two main crates: a standard Rust application crate and a no_std shader crate. The shader crate contains all the GPU entry points and shared data structures. A significant advantage of this setup is the seamless sharing of Rust types between CPU and GPU code. By adding the shader crate as a dependency to the main crate, developers can define structs and enums once, ensuring type consistency and eliminating common sources of API mismatches.
The build process for Rust GPU is orchestrated by cargo gpu install. When a developer runs cargo build, the main crate's build script invokes cargo gpu install. This tool is responsible for:
- Identifying the correct Rust nightly version required by the Rust GPU compiler backend.
- Downloading and setting up the custom Rust GPU backend.
- Invoking
rustcwith this custom backend to compile the shader crate for the SPIR-V target. This is in addition to the standard compilation of the shader crate for the CPU.
The Rust GPU backend forks the data stream from rustc's codegen process, performing custom transformations to generate valid SPIR-V bytecode.
A core challenge addressed by Rust GPU is ensuring consistent memory layouts. Traditional graphics APIs use specific memory layouts like std430 or std140, which have peculiar rules (e.g., vec3 having a size of 12 bytes but an alignment of 16 bytes). These layouts are often incompatible with standard C or Rust #[repr(C)] layouts, leading to padding and alignment issues. Rust GPU bypasses this by simply asking rustc for the layout of a type and using that directly. This means that a #[repr(C)] struct in Rust will have the same layout on both CPU and GPU, provided care is taken with machine-dependent types like usize (which is 32-bit on the Rust GPU platform). The project also looks forward to initiatives like repr(Cranberry), which aims to provide stable, optimized layouts across compilers, potentially resolving current limitations with #[repr(rust)] (which can change between compiler invocations) and supporting more complex Rust features like Option types.
The SPIR-V environment itself imposes severe constraints:
- No Operating System: Shaders run directly on hardware, without OS services.
- No Allocator: Dynamic memory allocation (like
VecorHashMap) is not natively supported. - No Recursion: The architecture is typically register-based without a stack, making recursion impossible.
- Typed Memory: A compiler's "hell," requiring careful transformation from untyped Rust memory to explicit SPIR-V types.
- Logical Pointers: This is the most significant hurdle. SPIR-V logical pointers are abstract; they have no physical size, numeric value, or support for arithmetic. They can only be created from existing objects and cannot be stored in other objects. This means:
- No slicing of slices (requires pointer arithmetic).
- No enums of values (enum tags and data require offsets).
- No use of Rust's core pointer API (which is pervasive).
- No creating references to struct members (requires offsets).
- Crucially, no passing pointers as function arguments (this is illegal in SPIR-V Vulkan).
To overcome the "no pointers as function arguments" limitation, Rust GPU employs post-link legalization. During initial rustc codegen, Rust GPU emits SPIR-V that does contain references as function arguments, even though this is illegal. However, because shaders operate under a closed-world assumption (all code is known at compile time, no dynamic linking), a custom SPIR-V linker can analyze the entire shader. This linker performs a "post-link legalization" pass, primarily by inlining any function that receives a pointer argument. By inlining, the pointer is no longer "passed" across a function boundary, making the resulting SPIR-V legal. This approach leverages the flexibility of having a custom linker.
For more general pointer semantics, especially for heap-like structures, Rust GPU uses quasi-pointer emulation (qpointer). This technique tags pointers with metadata indicating their origin (e.g., a buffer, a constant) and an offset. When a "pointer" is accessed, the system inspects the tag to determine where to load the actual data from. For example, a conditional branch that takes a reference to either a buffer or a constant would be transformed into code that tags the "pointer" accordingly. Upon dereference, a match statement on the tag directs the load operation. This is effectively a form of generic address space emulation. With clever constant propagation and range analysis, the overhead can be minimized, especially when address spaces are not mixed. This qpointer system is robust enough to enable basic alloc functionality on the GPU, allowing for dynamic data structures like Vec (using a custom bump allocator), although allocations cannot currently be directly sent back to the CPU due to the custom pointer representation.
A critical issue arises with the representation of these quasi-pointers in memory, particularly when using Box<Vec<T>>. Rust expects pointers to have a specific size (e.g., usize). However, qpointer needs to store both a tag and an offset. The current solution involves pointer tagging, where the tag is stored in the higher bits of the pointer. This is problematic because the Rust GPU platform's usize is 32-bit (due to WebGPU's lack of 64-bit support), meaning the addressable range for buffers (typically 4GB) must be significantly reduced to accommodate the tag (e.g., down to 1GB or 512MB). This is a substantial limitation for large-scale GPU applications. Alternative solutions explored include the CHERI platform (hardware-assisted capability pointers), emulating 64-bit pointers with 32-bit operations (which WGSL doesn't support), or restricting pointers to local variables.
Finally, Rust GPU targets Vulkan shaders, which offer a broad ecosystem and support graphics and compute. This contrasts with other SPIR-V targets, such as those in LLVM or Zig, which often focus on OpenCL kernels. OpenCL kernels support physical pointers with address spaces, which is simpler but limits them to compute-only tasks and requires separate driver installations. Rust GPU's choice of Vulkan shaders, despite the complexity of logical pointers, offers wider applicability and the ability to transpile to WGSL for web deployment.
Demo / Proof of Concept
▶ Watch: Tiny Wasm: Stress testing GPU drivers with 90MB shaders (31:00)
The talk features several demonstrations highlighting the capabilities and current limitations of Rust GPU, particularly concerning memory management and general-purpose programming constructs.
The first practical demonstration showcases the ability to use Rust's alloc crate on the GPU, thanks to the quasi-pointer emulation (qpointer) system. Firestar99 illustrates this by creating a Vec<u32> on the GPU. Since GPUs typically lack a system allocator, a custom bump allocator is implemented and used. The demo shows a Vec initialized with [1, 2, 3], and its memory representation is visualized, clearly showing the allocated values. When a 4 is pushed into the Vec, which exceeds its initial capacity, the bump allocator performs a new allocation, doubles the capacity, and copies the elements, demonstrating dynamic resizing on the GPU. The speaker notes that while alloc works, these GPU-side allocations cannot currently be directly sent back to the CPU as Rust types like Box or Vec due to the custom pointer representation. Raw buffer dumps are still possible, but the semantic information of the Rust types is lost during transfer.
A more complex scenario is presented with Box<Vec<T>>, which further exposes the challenges of pointer representation. When a Vec is wrapped in a Box, the Box itself contains a pointer. In the Rust GPU environment, this pointer must encode both the memory offset and the qpointer tag. As the platform's usize is 32-bit (a constraint imposed by WebGPU's lack of 64-bit arithmetic support), storing the tag in the higher bits of the pointer significantly reduces the addressable memory range. This means that typical 4GB buffer limits in graphics APIs might be cut down to 1GB or even 512MB to make space for the tag, posing a severe limitation for applications requiring large memory buffers.
The most ambitious proof-of-concept involves compiling Tiny WASM, a small embedded WebAssembly interpreter, to SPIR-V using Rust GPU. This serves as a stress test, as Tiny WASM utilizes a wide range of Rust features, including recursion and function pointers (which require additional emulation). The result of compiling Tiny WASM was a staggering 90 megabytes of SPIR-V. For context, typical shader binaries are measured in kilobytes, with complex ones perhaps reaching a few megabytes with debug info. This massive binary size overwhelmed GPU drivers. The demo concluded with a screenshot showing a process consuming over 1 terabyte of memory, going deep into swap, and ultimately crashing due to out-of-memory errors, without ever finishing compilation or execution. This dramatic outcome highlights a critical assumption made by current GPU drivers: that shaders are inherently small and can be extensively inlined. When faced with a complex, large program like Tiny WASM, this inlining strategy leads to an exponential explosion in code size, making it unmanageable for existing driver architectures.
Defensive Implications
▶ Watch: Q&A: Comparing Rust GPU with NVIDIA's CUDA Oxide (43:00)
While this talk is not directly about security, the advancements in Rust GPU have significant defensive implications for the reliability and correctness of GPU programming. Traditional GPU development, characterized by separate CPU and GPU languages, is inherently prone to a class of errors known as API mismatches. These occur when the CPU code (e.g., defining a data structure) and the GPU code (e.g., accessing that data structure) have differing assumptions about memory layout, padding, or type interpretation. Such mismatches often lead to subtle, hard-to-debug issues like corrupted data, incorrect computations, or even crashes, which are difficult to diagnose in a black-box GPU environment.
Rust GPU directly addresses this by enabling a unified language workflow. By writing both CPU and GPU code in Rust, developers can define shared data structures in a single, common crate. Rust's strong type system then ensures that these structures are interpreted consistently across both host and device. Specifically, leveraging #[repr(C)] guarantees a predictable memory layout, side-stepping the complex and often error-prone layout rules (like std430 or std140) of traditional shader languages. This significantly reduces the surface area for layout-related bugs, acting as a powerful defensive measure against common programming errors that plague cross-language GPU development.
Furthermore, Rust's inherent memory safety features, such as its borrow checker and ownership model, extend to the GPU code. While the current implementation has to emulate certain pointer semantics due to SPIR-V's logical pointer model, the goal is to bring as much of Rust's safety guarantees as possible to the GPU. This means that issues like use-after-free, buffer overflows (within the bounds of the qpointer emulation), or data races (in concurrent compute shaders, though not explicitly detailed in the talk) could theoretically be caught by the Rust compiler or prevented by its design patterns, leading to more robust and less exploitable GPU code.
In essence, Rust GPU provides a "defense in depth" against programming errors by:
- Reducing cognitive overhead: Developers work in one language, reducing context-switching errors.
- Enforcing type safety: Rust's compiler catches many errors at compile time that would typically manifest as runtime bugs in less-safe languages.
- Ensuring memory layout consistency:
#[repr(C)]and the custom compiler backend prevent API mismatches related to data structure packing and alignment.
While the project is still evolving and faces challenges with full Rust feature parity and driver limitations (as seen with the Tiny WASM demo), its fundamental approach offers a compelling path towards more secure and reliable GPU programming, by mitigating the risk of entire classes of bugs at the architectural and language level.
Key Takeaways
- Unified Language for GPU Development: Rust GPU enables writing both CPU and GPU code in Rust, eliminating the need for separate shader languages and simplifying the development workflow for graphics and compute applications.
- Elimination of API Mismatches: By sharing Rust types and leveraging
#[repr(C)]for consistent memory layouts, Rust GPU directly addresses the common and frustrating problem of API mismatches between host and device code. - Innovative Compiler Engineering: The project utilizes a custom
rustcbackend and a custom SPIR-V linker to overcome fundamental architectural limitations of SPIR-V, such as its logical pointer model and lack of standard library support, through techniques like post-link legalization and quasi-pointer emulation (qpointer). - Enabling Advanced Rust Features:
qpointerallows for advanced Rust features like basic dynamic memory allocation (alloccrate with custom allocators) on the GPU, pushing the boundaries of what's traditionally possible in shader environments. - Practical Limitations and Driver Assumptions: Stress tests, such as compiling Tiny WASM, reveal that current GPU drivers and hardware are optimized for small shaders. Large, complex programs compiled to SPIR-V can lead to massive binary sizes (e.g., 90MB), causing drivers to crash due to excessive inlining and memory consumption.
- Strategic Target: Vulkan Shaders: Rust GPU specifically targets Vulkan shaders, which support both graphics and compute, offering broad device compatibility and the potential for transpilation to WGSL for web platforms, distinguishing it from OpenCL-focused SPIR-V compilers.
About the Speaker(s)
Firestar99, the speaker for this GPN24 talk, began his career as a graphics programmer, specializing in creating GPU-driven applications. His journey into the world of Rust began around 2022, leading him to discover the Rust GPU project. Intrigued by its potential, he chose to base his master's thesis on Rust GPU. During this period, he found himself needing additional features, which led him to contribute to the compiler, eventually submitting several Pull Requests. His active involvement and contributions led to an invitation to become a maintainer for the project. Firestar99 is now commercially involved in building and improving Rust GPU, a role that has transitioned him from a graphics programmer to a compiler developer. He candidly admits to being "compiler dev now and not a graphics programmer anymore," highlighting his deep immersion in the intricate world of compiler internals and low-level GPU programming. His unique perspective, combining practical graphics development experience with compiler-level expertise, provides valuable insight into the challenges and innovations of the Rust GPU project. He also briefly touches upon the history of Rust GPU, mentioning its origins with Embark Studios and Traverse Research, and the subsequent community fork that he joined and now helps maintain.
Reviews
Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT
This is real compiler engineering work presented by someone knee-deep in the codebase. Firestar99 walks through genuinely hard problems—SPIR-V's logical pointer model is a nightmare, and the qpointer emulation approach is clever. The 90MB Tiny WASM binary that crashes drivers is the kind of honest 'here's where we hit the wall' moment you rarely see.
Heather Calloway (CISO) — SOLID
Deep compiler engineering talk about making Rust compile to GPU shaders. Technically impressive work on a hard problem, but this is language tooling for graphics programmers — no enterprise security relevance. File under 'interesting things happening in Rust ecosystem' if you track that.