Challenges in Decompilation and Reverse Engineering of CUDA-based Kernels
Nicolò Altamura (Security Engineer · emproof)
RE//verse 2026 · Day 1 · Coral Ballroom A
Overview
In this insightful talk at RE//verse, Nicolò Altamura, a Security Engineer at emproof, delved into the complex and often opaque world of reverse engineering CUDA-based kernels. The presentation highlighted the significant hurdles faced when attempting to decompile and analyze code designed for NVIDIA's highly parallelized GPU architectures. As modern applications increasingly offload computationally intensive tasks to GPUs, understanding the underlying code executed on these devices becomes paramount for security analysis, software verification, and malware detection.

Key moments
- 0:00 Introduction: Why CUDA reverse engineering matters
- 4:00 CUDA compilation: PTX vs. SAS and fat binaries
- 7:50 PTX Ninja: Architecture plugin for PTX parsing
- 10:30 SAS decompilation pipeline and its challenges
- 12:10 Discovering SAS instruction specification using binary diffing
- 16:00 Synthesizing concrete instruction semantics with a debugger oracle
- 20:20 Future research: Data flow and type inference challenges
Challenges in Decompilation and Reverse Engineering of CUDA-based Kernels
Speakers: Nicolò Altamura, Security Engineer, emproof
Conference: RE//verse
YouTube: https://www.youtube.com/watch?v=ns5jFuEdeFg
Overview
In this insightful talk at RE//verse, Nicolò Altamura, a Security Engineer at emproof, delved into the complex and often opaque world of reverse engineering CUDA-based kernels. The presentation highlighted the significant hurdles faced when attempting to decompile and analyze code designed for NVIDIA's highly parallelized GPU architectures. As modern applications increasingly offload computationally intensive tasks to GPUs, understanding the underlying code executed on these devices becomes paramount for security analysis, software verification, and malware detection.
The core challenge stems from CUDA's unique parallel programming paradigm and NVIDIA's proprietary nature, which results in a lack of formal specifications for its instruction set architecture (ISA). This opacity creates a significant "missed opportunity" in binary analysis, making it difficult for researchers and engineers to inspect, understand, and secure GPU-accelerated software. Altamura's work directly addresses this gap, presenting both a practical tool for Parallel Thread Execution (PTX) code and a systematic methodology for tackling the more elusive Streaming Assembler (SAS).
The talk not only introduced the fundamental concepts of CUDA architecture and its dual-stage compilation process but also detailed a pipeline for decompiling GPU code. Altamura demonstrated a custom architecture plugin, PTX Ninja, for Binary Ninja, showcasing its ability to parse and analyze PTX code. More critically, he outlined an innovative approach to reverse engineer the undocumented SAS, employing statistical testing and debugger-as-oracle techniques to synthesize instruction formats and concrete semantics. This work is crucial for advancing the capabilities of reverse engineering tools and methodologies in the rapidly evolving landscape of GPU computing.
Background
▶ Watch: Introduction: Why CUDA reverse engineering matters (0:00)
The proliferation of high-performance computing has led to widespread adoption of GPUs for tasks beyond graphics rendering, particularly in scientific simulations, machine learning, and data analytics. NVIDIA's CUDA (Compute Unified Device Architecture) platform is a dominant force in this domain, providing a parallel computing platform and programming model that leverages the immense processing power of GPUs. Unlike traditional CPUs, which excel at sequential tasks, GPUs are designed with an array of Streaming Multiprocessors (SMs), each containing multiple cores and Arithmetic Logic Units (ALUs), capable of executing thousands of threads concurrently.
This host and device paradigm defines CUDA programming: the CPU (host) manages the overall application, while computationally intensive routines, known as kernels, are offloaded to the GPU (device). The host sends data to the device, the GPU executes the kernels in parallel, and then the host retrieves the results. While incredibly efficient for specific workloads, this parallel architecture introduces significant complexities for reverse engineering. Traditional decompilers and analysis tools are built with assumptions rooted in sequential CPU architectures, making them ill-suited for the highly parallel, often predicated, and register-rich environment of GPUs.
The CUDA compilation pipeline further complicates matters due to its dualism. Source code (e.g., CUDA C++) is first compiled into PTX (Parallel Thread Execution), a text-based, low-level virtual ISA. PTX is designed to be forward-compatible, targeting all CUDA architectures, and is translated just-in-time (JIT) into SAS (Streaming Assembler)—the actual machine code executed on a specific GPU device. While PTX has public documentation, SAS is proprietary, with NVIDIA providing little to no public details about its instruction set. This opaque nature of SAS is the primary hurdle for deep analysis, as many distributed kernels are available only in this device-specific, undocumented format. The lack of formal specifications for SAS means that binary analysts face a black box, hindering efforts in vulnerability research, intellectual property protection, and advanced malware analysis targeting GPU code.
Key Findings
▶ Watch: PTX Ninja: Architecture plugin for PTX parsing (7:50)
Nicolò Altamura's presentation unveiled several critical findings and methodologies addressing the challenges of CUDA kernel reverse engineering:
- PTX Ninja Plugin: A key contribution is the development of PTX Ninja, an architecture plugin for Binary Ninja. This tool effectively parses and represents PTX (Parallel Thread Execution) code, translating its text-based, RISC-like instructions into Binary Ninja's intermediate representation. This allows for immediate, human-readable analysis of PTX kernels, overcoming the initial hurdle of understanding the virtual ISA, including its use of unlimited virtual registers and predicated instructions.
- SAS Instruction Specification Mining: For the undocumented SAS (Streaming Assembler), Altamura proposed and demonstrated a systematic methodology to synthesize its instruction format. By leveraging binary diffing and statistical testing—placing identical instructions in different kernels and observing byte patterns—he discovered that SAS instructions for the Volta architecture are 16 bytes long. These 16 bytes are split into two distinct components: 8 bytes for the concrete instruction semantics (opcode, operands) and 8 bytes for context-sensitive information or control codes, which relate to pipeline and register usage and are often skipped by disassemblers.
- Concrete Semantic Synthesis via Debugger Oracle: To understand the actual behavior of SAS instructions, Altamura introduced a novel approach: treating a debugger as an "oracle." This involves feeding an initial GPU state, setting a breakpoint, single-stepping over an instruction, and then observing the resulting final state. By comparing initial and final states, a possible concrete semantic for the instruction can be synthesized. While acknowledging limitations (e.g., predicated instructions, synchronization primitives), this method provides a practical way to reverse engineer the instruction's effects without formal documentation.
- Challenges in Control Flow and Data Flow Analysis: The talk highlighted specific difficulties in building accurate Control Flow Graphs (CFGs) and performing Data Flow Analysis for CUDA kernels. Identifying basic block leaders is complicated by synchronization instructions. More profoundly, tracking values and recognizing high-level data structures like 1D/ND arrays (matrices and tensors) is exceptionally difficult. This is because arrays are often accessed via row pointers and complex index expressions, requiring computationally expensive relational-based analysis and range analysis (e.g., using polyhedra) to accurately determine boundaries and relationships.
- CUDA-Specific Control Flow Peculiarities: Altamura identified "weird loops" in SAS code related to parameter passing and return values. Unlike CPU functions, CUDA kernels often lack a "real return" mechanism, instead signaling task completion to the host, which then handles runtime termination. Furthermore, CUDA's architecture imposes constraints on kernel complexity (to avoid divergence), which paradoxically simplifies control flow restructuring compared to general-purpose CPU code.
Technical Deep Dive
▶ Watch: SAS decompilation pipeline and its challenges (10:30)
The technical core of the presentation revolved around dissecting CUDA's architecture, its compilation process, and the intricate methodologies required for reverse engineering both PTX and SAS.
CUDA Architecture and the Host-Device Paradigm
At a fundamental level, CUDA leverages the highly parallel nature of GPUs. A GPU is composed of an array of Streaming Multiprocessors (SMs), each equipped with multiple cores and ALUs. These cores can execute operations in parallel via numerous threads. CUDA provides synchronization primitives, instruction schedulers, specialized ALUs (e.g., for floating-point operations), and hardware caches across the execution pipeline to manage this parallelism efficiently.
The host and device paradigm dictates that the CPU (host) orchestrates the overall application, while the GPU (device) executes specialized routines called kernels. The typical workflow involves:
- Host sends data to the device.
- GPU executes the kernel(s) on the data.
- Host receives the processed data or results from the device.
This offloading of heavy computations to the GPU is where the performance gains lie, but also where the reverse engineering challenges begin.
CUDA Compilation Pipeline: PTX vs. SAS
The CUDA compilation process is complex due to the need to target a wide range of GPU architectures. This results in a dual-stage approach:
- PTX (Parallel Thread Execution): This is a text-based, low-level virtual instruction set architecture. It serves as an intermediate representation, targeting all CUDA architectures. PTX is designed for forward compatibility; it gets translated just-in-time (JIT) into SAS before execution on a specific GPU. NVIDIA provides public documentation for PTX, which significantly aids initial analysis. PTX instructions are typically RISC-like, operate on an unlimited number of virtual registers, and can be predicated (i.e., their execution depends on a boolean predicate register). A PTX instruction comprises an opcode, a type (e.g.,
U32for 32-bit unsigned integer), and optional modifiers. Operands can be immediate values, virtual registers, special read-only registers (for thread context), or memory addresses (e.g., shared memory).
- SAS (Streaming Assembler): This is the actual machine code executed on a specific CUDA device. Unlike PTX, SAS is highly device-specific and undocumented by NVIDIA. It contains low-level microarchitectural details, including pipeline information. Many CUDA kernels are distributed solely as SAS executables (within cubin files), making its reverse engineering crucial for comprehensive analysis.
PTX Decompilation with PTX Ninja
Leveraging the public documentation for PTX, Altamura developed PTX Ninja, an architecture plugin for Binary Ninja. This plugin translates raw PTX instructions into Binary Ninja's intermediate representation (IR), allowing the platform's existing analysis capabilities (e.g., control flow, data flow) to be applied. For example, a simple mov.u32 %R25, %R0; instruction in PTX is lifted to an assignment R25 = R0 with a 32-bit unsigned integer type. The plugin handles various PTX opcodes, types, and modifiers, facilitating a clear, high-level understanding of PTX kernels. The talk demonstrated PTX Ninja's effectiveness by showing the decompilation of complex kernels, including matrix multiplication and a transformer attention algorithm, making their logic readily apparent.
SAS Decompilation Pipeline and Challenges
The primary focus of Altamura's research is the decompilation of SAS, which presents a "black box" problem due to the lack of public specifications. The proposed pipeline for SAS decompilation follows the standard decompiler model:
- Executable Parsing: Extract instructions from cubin files (ELF files with a CUDA-specific machine architecture).
- Instruction Extraction (Disassembly): This is the first major hurdle. NVIDIA provides tools like
cuobjdumporNV disasm, but these are black-box binaries that do not reveal the underlying instruction bit fields. - Intermediate Representation (IR) Translation: Convert raw SAS instructions into an IR.
- Control Flow Graph (CFG) Construction: Group instructions into basic blocks.
- Data Flow Analysis: Track values and dependencies.
- High-Level Control Flow Restructuring: Identify loops, conditionals, etc.
- Type Inference: Deduce data types.
- Source Code Generation: Produce human-readable code.
Mining SAS Instruction Specifications
To overcome the lack of SAS documentation, Altamura devised a systematic approach:
- Binary Diffing: By compiling minimal CUDA kernels with slight variations and observing the resulting SAS bytes, he could deduce the instruction format. This revealed that for the Volta architecture, each SAS instruction is 16 bytes long.
- Instruction Structure: These 16 bytes are divided into two 8-byte components:
- The first 8 bytes encode the concrete semantics (opcode, operands, modifiers).
- The second 8 bytes contain control codes, which are context-sensitive and relate to the GPU's pipeline and register usage. Disassemblers typically skip these control codes, focusing on the semantic part.
- Instruction Enumeration: By systematically mutating byte buffers in minimal SAS binaries and dumping the instruction text using
NV disasm, Altamura was able to enumerate a large number of opcodes and identify operands, including constant banks (constant memory) and uniform registers (shared at the warp level), which are distinct from PTX's virtual registers.
Synthesizing Concrete Semantics
After identifying the instruction format, the next challenge is understanding what each instruction does. Direct intuition is insufficient for complex opcodes. Altamura's solution involves treating a debugger (e.g., a CUDA debugger) as an oracle:
- Initial State Capture: Record the GPU's register and memory state before an instruction.
- Single-Step Execution: Execute only the target SAS instruction.
- Final State Capture: Record the GPU's state after execution.
- Semantic Synthesis: Compare the initial and final states to infer the instruction's effect (e.g.,
R1 = R2 + R3for anaddinstruction).
This process, while powerful, has limitations. Predicated instructions might not alter the state if their predicate is false, leading to "no effect" observations. Synchronization instructions and special registers also pose challenges due to their complex side effects.
Control Flow Graph and Data Flow Analysis Challenges
Constructing a CFG for SAS is generally feasible, as basic blocks can be identified by instructions that modify the program counter (e.g., jumps) and by synchronization instructions, which often delineate block boundaries. Static Single Assignment (SSA) form, a standard IR optimization, is also a well-understood problem.
However, data flow analysis for CUDA kernels is particularly difficult and computationally expensive:
- Relational Analysis: CUDA's paradigm heavily relies on 1D and ND arrays (matrices, tensors). These are accessed via row pointers and complex index expressions, making it hard to track individual elements or reconstruct array structures. Relational-based analysis (e.g., using polyhedra theory) is often required to understand relationships between memory accesses, but such analyses are very costly.
- Range Analysis: Determining the possible values or boundaries of array indices (e.g.,
0 to 3forR2andR3in a 3D access) is crucial but also computationally intensive. - Type Inference: This problem is tightly coupled with data flow analysis; without understanding data flow, inferring high-level types like
matrixortensoris difficult.
Control Flow Restructuring and Type Inference
Paradoxically, control flow restructuring (identifying high-level constructs like if-else and loops) can be somewhat simpler for CUDA kernels compared to general CPU code. This is because CUDA developers are incentivized to write simpler control flows to avoid divergence, a condition where threads within a warp (a group of threads executing in lockstep) take different execution paths, leading to performance degradation. The simpler the control flow, the faster the kernel. Despite this, the final step of rewriting the decompiled output into a human-understandable format that accurately expresses relationships between matrices and tensors remains an open research problem.
Demo / Proof of Concept
▶ Watch: Synthesizing concrete instruction semantics with a debugger oracle (16:00)
During the talk, Nicolò Altamura provided a clear demonstration of the PTX Ninja plugin integrated within Binary Ninja. This proof of concept illustrated the immediate benefits of having a robust parser for PTX (Parallel Thread Execution) code.
The demo began by showcasing a simple kernel designed for matrix multiplication. Altamura highlighted how PTX Ninja successfully parsed the PTX assembly, presenting it in a structured and analyzable format within Binary Ninja's interface. He specifically pointed out instructions involving multiple multiplications and additions, demonstrating how the plugin translates these low-level PTX operations into a more understandable intermediate representation, making the underlying matrix multiplication logic evident.
A more complex example was then presented: a kernel implementing a portion of the transformer attention mechanism, a crucial algorithm in Large Language Model (LLM) training. This kernel involved intrinsic CUDA calls and more intricate logic, including dot products, scaling, and matrix operations. The PTX Ninja plugin again proved its capability, displaying the structured code and allowing for the identification of these advanced computational patterns.
This demonstration effectively validated the utility of PTX Ninja in overcoming the initial hurdle of parsing and analyzing PTX code, providing a foundation for deeper security analysis and software understanding. While the talk extensively discussed the challenges and methodologies for SAS (Streaming Assembler) decompilation, the live demo focused on the already-implemented and functional PTX analysis, serving as a tangible first step towards the broader goal of full CUDA kernel reverse engineering.
Defensive Implications
▶ Watch: Future research: Data flow and type inference challenges (20:20)
The work presented by Nicolò Altamura has profound defensive implications for securing GPU-accelerated applications. As more critical software, including machine learning models, cryptographic operations, and even malware, leverage CUDA kernels, the ability to reverse engineer and understand this code becomes indispensable.
- Malware Analysis and Detection: GPU-aware malware is an emerging threat. Without the ability to decompile SAS kernels, security analysts are effectively blind to malicious code executing on the GPU. Altamura's methodologies for mining SAS instruction sets and synthesizing concrete semantics provide a pathway to build tools that can disassemble and analyze these kernels, enabling the detection of malicious activities, obfuscation techniques, or exfiltration attempts occurring at the GPU level. This capability is crucial for identifying novel GPU-specific attack vectors.
- Vulnerability Research and Exploitation: Understanding the precise behavior of CUDA kernels is vital for identifying vulnerabilities. Flaws in memory access patterns, synchronization primitives, or arithmetic operations within kernels could lead to severe security issues such as data corruption, privilege escalation, or denial-of-service. By being able to inspect the actual machine code (SAS), researchers can uncover subtle bugs that might be missed in higher-level source code reviews or PTX analysis. This could lead to the discovery of new CVEs affecting GPU drivers, CUDA libraries, or specific GPU-accelerated applications.
- Software Verification and Compliance: For high-assurance systems, verifying the correctness and security of all code components, including those running on GPUs, is paramount. The lack of SAS decompilation tools makes formal verification of GPU kernels extremely challenging. Altamura's approach lays the groundwork for creating tools that can translate SAS into an analyzable intermediate representation, facilitating static analysis, formal verification, and compliance checks for critical GPU workloads in industries like automotive, aerospace, and finance.
- Intellectual Property Protection: For software vendors, the ability to analyze and protect proprietary algorithms implemented in CUDA kernels is crucial. Reverse engineering capabilities can help identify instances of intellectual property theft or unauthorized use of patented algorithms by analyzing competitor binaries. Conversely, it can also help vendors understand how their own GPU code might be vulnerable to reverse engineering attempts, leading to better obfuscation or protection strategies.
- Enhanced Incident Response: During a security incident involving systems with GPUs, understanding what code was executed on the GPU is critical for forensic analysis. Current limitations mean that GPU execution traces are often opaque. By providing the means to analyze SAS, this research enables more comprehensive forensic investigations, helping to reconstruct attack chains and determine the full impact of a breach.
In essence, Altamura's work transforms the GPU from a security blind spot into an observable and analyzable component, empowering defenders with the necessary tools and methodologies to secure the increasingly GPU-dependent software ecosystem.
Key Takeaways
- CUDA Kernel Reverse Engineering is Critical but Challenging: The shift towards GPU-accelerated computing for high-performance applications necessitates robust reverse engineering capabilities, but NVIDIA's proprietary SAS (Streaming Assembler) ISA poses a significant "black box" problem.
- PTX Analysis is Achievable with Tools like PTX Ninja: The public documentation for PTX (Parallel Thread Execution) allows for the development of effective tools, such as the demonstrated PTX Ninja plugin for Binary Ninja, to parse and analyze this virtual ISA.
- SAS Instruction Mining Requires Innovative Methodologies: Without formal specifications, techniques like binary diffing and statistical testing are essential to synthesize the instruction format, revealing crucial details like the 16-byte instruction length (8 bytes for concrete semantics, 8 for control codes) in Volta architecture.
- Debugger-as-Oracle Approach for Semantic Synthesis: Inferring the concrete behavior of undocumented SAS instructions can be achieved by treating a debugger as an oracle, observing state changes after single-stepping through instructions, despite challenges like predication and synchronization.
- Data Flow Analysis Remains a Significant Bottleneck: Accurately tracking values, especially for 1D/ND arrays (matrices, tensors) accessed via row pointers and complex index expressions, requires computationally expensive relational-based analysis and range analysis, which are not readily available in current tools.
- Understanding CUDA's Microarchitecture is Key to Decompilation: Peculiarities like the host-device paradigm, parameter passing mechanisms, and the impact of divergence on control flow complexity must be factored into any successful CUDA decompilation pipeline.
About the Speaker(s)
Nicolò Altamura is a Security Engineer at emproof, where he is involved in building software protection for embedded devices. In his free time, he pursues interests in obfuscation techniques. Nicolò recently graduated from the University of Verona approximately six months prior to giving this talk. His research focuses on the challenges of reverse engineering and decompilation, particularly in complex architectures like CUDA.
Reviews
Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT
Altamura takes a sledgehammer to NVIDIA’s undocumented SAS instruction set and actually makes a dent. Unpacking Volta’s 16-byte instruction format via binary diffing is the exact kind of masochistic, low-level RE work we need right now as GPU-aware malware scales. He drops a functional PTX lifter for Binja and outlines a realistic, if painful, path for fully reversing bare-metal CUDA.
Heather Calloway (CISO) — MUST SEE
This is deep technical research on reverse-engineering undocumented NVIDIA GPU microcode. It exposes a massive telemetry blind spot in our AI and high-compute infrastructure that current security tools cannot penetrate. If your business relies on heavy ML workloads, your architecture team needs to understand this exposure.