Déjà Vu in Linux io_uring: Breaking Memory Sharing Again After Generations of Fixes

Déjà Vu in Linux io_uring: Breaking Memory Sharing Again After Generations of Fixes

Speakers: Chih-Yen Chang (Pumpkin), Security Researcher, DEVCORE

Conference: Hexacon

YouTube: https://www.youtube.com/watch?v=Ry4eOgLCo90

Overview

This talk, presented by Chih-Yen Chang, also known as Pumpkin, from DEVCORE, delves into a critical race condition he discovered within the Linux kernel's io_uring subsystem, identified as CVE-2025-2136. The vulnerability is particularly noteworthy because it exploits a memory sharing mechanism that has been the subject of multiple previous fixes, yet continues to harbor subtle flaws. The "déjà vu" in the title aptly captures the recurring nature of these issues, where attempts to address one problem inadvertently create conditions for another, highlighting the profound complexities of concurrent memory management in the kernel.

Pumpkin meticulously dissects the evolution of memory sharing vulnerabilities in io_uring, tracing a lineage of three prior bugs and their respective patches that ultimately paved the way for his discovery. His research demonstrates how even seemingly benign "dummy operations" or architectural shifts, such as replacing a mutex with an RCU (Read-Copy Update) lock, can introduce dangerous race windows when combined with intricate object lifecycle management. The culmination of this intricate dance of fixes and new vulnerabilities is a Use-After-Free (UAF) on the io_buffer object, providing a powerful primitive for arbitrary kernel memory mapping with read/write permissions, leading to full root privilege escalation.

The significance of this work extends beyond a single CVE. It serves as a potent reminder of the inherent challenges in securing high-performance kernel subsystems like io_uring, which prioritize efficiency through complex memory sharing and asynchronous operations. The talk not only details the technical specifics of the exploit but also offers valuable insights into the methodologies for identifying such deep-seated concurrency bugs and the broader implications for kernel security and defensive strategies.

Background

▶ Watch: Introduction to io_uring race and talk agenda (0:00)

The io_uring subsystem, introduced in the Linux kernel in 2019, was designed to revolutionize I/O operations by significantly reducing overhead. It achieves this by allowing processes to submit I/O requests asynchronously and leverage kernel threads as workers, eliminating the need for processes to block and wait for I/O completion. A cornerstone of its performance optimization is an advanced memory sharing mechanism, which aims to eliminate redundant memory copies between kernel and user space, enabling zero-copy data transfer.

From a high-level perspective, io_uring can be understood through three primary components: the IO command, the command worker, and memory sharing. Processes encapsulate I/O requests into io_uring command formats, which are then validated by handlers and dispatched to background kernel worker threads. The memory sharing component, the focus of this talk, provides two main types of shared memory:

1. Ring buffers: Used for the submission queue (SQ) and completion queue (CQ), facilitating the exchange of I/O requests and results.
2. Kernel buffers: These store user data directly, enabling zero-copy data transfer for I/O operations. This is the specific type of memory sharing that Pumpkin's research investigates.

Kernel buffers, managed by io_buffer objects, abstract io_uring's memory management. A process can register memory regions with an io_uring context, and these regions are then managed by io_buffer objects stored in an array within that context. When an I/O command specifies a buffer index, the kernel worker directly accesses the data in the associated io_buffer object's backing pages.

io_buffer objects themselves come in two primary flavors:

• Ring buffers: Represent large, contiguous memory regions. They can be created by registering anonymous pages (IORING_OP_REGISTER_BUFFERS) or by asking the kernel to allocate memory using the IOU_PBUF_RING_UNMAPPED flag. For kernel-allocated memory, the user space must explicitly map it via the mmap syscall before access.
• Provided buffers: Comprise non-contiguous buffers, created using the IORING_OP_PROVIDE_BUFFERS command. These can be added or removed using IORING_OP_REMOVE_BUFFERS.

Both types of buffers share the io_buffer_list structure, which includes crucial fields like is_mapped and is_unmapped to determine the buffer's type and state.

The complexity of io_uring's design, particularly its memory management, has made it a fertile ground for security vulnerabilities. A 2023 Google KCTF survey highlighted io_uring as a significant source of kernel vulnerabilities. Pumpkin's decision to focus on memory sharing was driven by its deep connection to memory mapping and page allocation primitives, which often lead to powerful exploitation primitives. He also noted that the use of "frags" to represent internal states within these structures frequently creates subtle corner cases, ripe for security issues when different states become entangled.

Key Findings

▶ Watch: IO buffer object details and data access flow (4:00)

Chih-Yen Chang's central discovery is CVE-2025-2136, a novel race condition in the Linux io_uring subsystem that leads to a Use-After-Free (UAF) vulnerability on io_buffer objects. This UAF, born from a delicate interplay of concurrent operations during an io_uring buffer "upgrade" and a simultaneous mmap handler execution, allows an attacker to map arbitrary kernel memory into user space with full read/write permissions. The ultimate consequence is a local privilege escalation to root.

The significance of CVE-2025-2136 lies in its "déjà vu" nature. It demonstrates that despite multiple prior attempts to secure io_uring's memory sharing — including deferred page release, the adoption of RCU locks to prevent deadlocks, and reference count updates to prevent premature object resets — fundamental concurrency challenges persist. Each previous fix, while solving an immediate problem, inadvertently introduced new conditions or overlooked subtle interactions that ultimately contributed to this latest vulnerability.

Specifically, the vulnerability arises from a race between two critical operations:

1. An "upgrade" operation, where an empty provided buffer is reused and reinitialized as a ring buffer. This operation, intended to optimize memory reallocation, includes a "dummy" step that forcefully resets the io_buffer object's reference count to one.
2. A concurrent mmap handler attempting to map this same io_buffer object (which, due to the race, temporarily appears mapable). The mmap handler correctly increments the reference count before use and decrements it afterwards.

The race window is extremely narrow but, when hit, results in the io_buffer object's reference count dropping to zero prematurely, leading to its freeing while still being referenced by the io_uring context. This UAF allows an attacker to reclaim the freed io_buffer object with controlled data, specifically manipulating its buff_ring field to point to an arbitrary kernel address. By subsequently mapping this reclaimed object, the attacker gains read/write access to that kernel memory, enabling powerful primitives like overwriting the core_pattern kernel variable to execute a malicious binary with root privileges.

Technical Deep Dive

▶ Watch: First memory sharing type: Ring Buffer registration (5:36)

The vulnerability CVE-2025-2136 is not an isolated flaw but rather the latest iteration in a series of security issues stemming from the intricate design of io_uring's memory sharing. To understand its root cause, it's essential to trace the evolution of three preceding bugs and their fixes.

1. CVE-2024-0522: Page Use-After-Free

The first vulnerability, CVE-2024-0522, highlighted a fundamental flaw in how io_uring managed mapped memory. When a process registered a ring buffer with the IOU_PBUF_RING_UNMAPPED flag and subsequently mapped it into user space using the mmap syscall, the kernel's remap_pfn_range function was used. This low-level function directly mapped physical addresses without updating the underlying page's reference count. Consequently, the io_buffer object was the sole entity managing the page's lifetime.

The problem arose when the io_buffer object was unregistered. The handler would release the io_buffer and its backing pages. However, the physical memory remained mapped into user space, accessible through the previous mmap region. This created a Use-After-Free (UAF) on the page itself.

The Fix: Developers introduced a deferred page release mechanism. A new linked list, io_buff_list, was added to the io_uring context structure. When an io_buffer object was released, its pages were no longer immediately freed but instead moved to this list, only to be freed when the entire io_uring context was destroyed. While this mitigated the immediate UAF, Pumpkin noted that it "probably just fixes the issue on the surface," implying it didn't fundamentally address the complex ownership and lifetime management.

2. Circular Lock Dependency (Deadlock)

While addressing CVE-2024-0522, another critical issue emerged: a circular deadlock between the io_uring context lock (a mutex) and the memory management (MM) lock.

• Process 1 (Registration): Acquires the io_uring context lock, then attempts to acquire the MM lock during a page fault (e.g., when initializing anonymous pages for a ring buffer).
• Process 2 (Mapping/Unmapping): For mmap or munmap operations on io_uring buffers, it would first acquire the MM lock, and then attempt to acquire the io_uring context lock.

If P1 held the io_uring lock and waited for the MM lock, while P2 held the MM lock and waited for the io_uring lock, a classic deadlock would occur.

The Fix: The mutex lock protecting the io_uring context was replaced with an RCU (Read-Copy Update) lock. RCU is a lock-free synchronization mechanism designed for performance, where readers don't block writers, and writers defer the release of old data structures until all readers have finished. This change effectively resolved the deadlock by allowing concurrent access to the io_uring context.

New Problem Introduced: While RCU prevented deadlocks and allowed deferred object release, Pumpkin immediately questioned, "could this change cause any problems?" The crucial insight was that RCU only guarantees that an object won't be freed while under protection; it does not prevent it from being modified concurrently. This opened the door for race conditions where an io_buffer object could be accessed and modified simultaneously.

3. CVE-2024-3580: Incorrect Memory Mapping

This vulnerability, discovered by Starlabs, was a direct consequence of the RCU change. It exploited a race between the unregister handler (which cleans up resources and resets io_buffer fields) and the mmap handler (which retrieves io_buffer fields for mapping).

The io_buffer_list structure contains a union field that can be interpreted as buff_ring (a pointer to shared memory) or buff_list (a list head for internal management). During the unregister process, if the io_buffer object's reference count dropped to zero, the handler would call init_list_head on the buff_list field to reset it. This function would modify the prev pointer within buff_list to point to the address of the io_buffer object itself.

The Race:

1. Process 2 (P2) starts mmaping an io_buffer object. It retrieves the object and passes initial checks.
2. Process 1 (P1) concurrently unregisters the same io_buffer object. Its reference count drops to zero, and the handler calls init_list_head on the buff_list union field. This modifies the buff_ring pointer (because it's a union) to point to the io_buffer object itself.
3. P2 then attempts to retrieve the buff_ring address to map. However, due to P1's concurrent modification, buff_ring now points to the io_buffer object's own address, not the intended backing pages.
4. The kernel incorrectly maps the page containing the io_buffer object into user space, instead of the data pages.

The Fix: Developers added a reference count update in the mmap handler. Before using an io_buffer object, the mmap handler would now attempt to take a reference. If the reference count had already dropped to zero (meaning the object was being reset or was about to be freed), the mapping would fail. This prevented the io_buffer object from being reset while it was being mapped.

The New Bug: CVE-2025-2136 (Race in io_uring Upgrade)

Despite the previous fixes — deferred page release, RCU protection, and mmap handler reference count checks — concurrent access to io_buffer objects was still possible, leading to CVE-2025-2136.

The vulnerability lies in the "upgrade" operation, a feature designed to reuse an empty provided buffer as a ring buffer to reduce memory reallocation overhead. This is implemented within the IORING_OP_REGISTER_BUFFERS handler.

Normally, a provided buffer cannot be mapped by mmap because its is_mapped flag is zero. Its reference count also typically stays at one (held by the io_uring context) because mmap operations are blocked.

The Race Scenario:

1. Process 1 (P1 - Upgrade): Initiates an IORING_OP_REGISTER_BUFFERS command to upgrade an empty provided buffer into a ring buffer.

• P1 retrieves the io_buffer object.
• P1 checks that the provided buffer is empty (has no sub-buffers).
• P1 then proceeds to reinitialize the io_buffer object as if it were a new ring buffer. Critically, as part of this reinitialization, P1 sets the is_mapped flag to one and forcefully sets the reference count to one without any prior checks.

1. Long Interrupt 1: A lengthy timer interrupt or other kernel operation occurs, suspending P1.
2. Process 2 (P2 - mmap): Concurrently attempts to mmap this same io_buffer object (which was originally a provided buffer).

• P2 retrieves the io_buffer object.
• P2 checks the is_mapped flag. Because P1 already set it to one, P2's check passes, allowing mmap to proceed on what should be an unmapable provided buffer.
• P2 then increments the io_buffer object's reference count (this is the reference count increment added to fix CVE-2024-3580). The refcount is now 2.

1. Long Interrupt 2: Another long interrupt occurs, suspending P2.
2. Process 1 (P1 - Resume Upgrade): P1 resumes its upgrade operation.

• P1 proceeds with the reinitialization, and crucially, again forcefully sets the reference count to one. This overwrites P2's increment, effectively "stealing" P2's reference.

1. Process 2 (P2 - Resume mmap): P2 resumes its mmap operation.

• P2 completes the mapping and then decrements the reference count (the reference P2 thought it held).
• Since P1 reset the refcount to 1, P2's decrement reduces it to 0.

1. Result: The io_buffer object's reference count drops to zero, triggering its release and freeing. However, P2 still holds a valid memory mapping to this now-freed io_buffer object (or rather, the kernel memory it now occupies), leading to a Use-After-Free (UAF).

This race condition is "extremely hard to hit" due to the very narrow time windows required for the two long interrupts to precisely interleave the critical steps.

Demo / Proof of Concept

▶ Watch: Rationale for focusing on io_uring memory sharing (8:50)

Pumpkin demonstrated the exploitation of CVE-2025-2136 in a kernel CTF environment. The exploit chain, designed to achieve arbitrary kernel memory read/write and ultimately root privileges, was broken down into six steps, which can be grouped into three main parts:

1. Reclaiming the Freed io_buffer Object

• Target Object: The freed io_buffer object, specifically a provided buffer, is allocated from kmem_cache_alloc_node (specifically the kmalloc-64 slab in the CTF environment).
• Reclamation: To reclaim this freed memory with attacker-controlled data, the exploit sprays msg_msg objects. These objects are also typically allocated from kmalloc-64, making them suitable for reclaiming the freed io_buffer slot.
• RCU Callback Delay: The kernel CTF environment uses tree_rcu, which freezes objects in batches. This means the freed io_buffer object can remain alive for about 5 seconds before being truly freed by the RCU callback. The exploit accounts for this by waiting approximately 5 seconds after triggering the race before attempting reclamation.

2. Winning the Race and Hijacking for Root

• Side Channel for Race Success: Triggering this race condition is difficult, so the exploit needs a reliable way to determine if the race was won. It leverages the return value of the mmap syscall. If the mmap call returns a valid pointer, it means the io_buffer object was still alive (reference count > 0), indicating the race failed. If mmap returns an error (specifically, because the atomic_inc_zero function failed, implying the reference count was already zero), the race was successful. The exploit retries the entire process until mmap fails.
• KASLR Bypass: In the kernel CTF environment, KASLR (Kernel Address Space Layout Randomization) is a hurdle. The exploit bypasses this by using the entry_B side channel to leak the kernel text address. This leaked address is crucial for targeting specific kernel variables.
• Arbitrary Kernel R/W: Once the race is won, the attacker has reclaimed the io_buffer object with a controlled msg_msg object. The buff_ring field within the io_buffer structure (now controlled by the msg_msg data) is manipulated to point to a desired kernel address, specifically the core_pattern kernel variable.
• Privilege Escalation: By mmaping this reclaimed object, the attacker gains read/write access to core_pattern. core_pattern dictates the path where core dump files are written. Overwriting it with the path to an attacker-controlled executable (e.g., /tmp/exploit_shell) and then triggering a segmentation fault (e.g., by accessing an invalid memory address) causes the kernel to execute the attacker's binary with root privileges.

3. Making the Race Stable

• Widening the Race Window: The inherent difficulty of hitting such a narrow race window required a stabilization technique. The exploit uses timer interrupts by creating a timer instance and queuing a large number of I/O events. This causes the timer interrupt handler to spend a significant amount of time iterating through its waiter list, effectively creating "long interrupts" that extend the critical race windows needed for the two processes (upgrade and mmap) to interleave precisely.
• Calculated Timeout Range: Instead of brute-forcing random timeouts, the exploit calculates a reasonable timeout range. This involves measuring the total execution time of the critical path and the overhead of a syscall, allowing for a more targeted and efficient sweep of timeout values.

In the demo video, Pumpkin showcased an alternative path to root: injecting shellcode into a kernel function to directly grab root credentials, switch namespaces, and escape a chroot jail. He noted that while the success rate in the kernel CTF environment was "pretty low," it surprisingly achieved an average success rate of around 30% in GitHub Actions environments, indicating the exploit's practical viability.

Defensive Implications

▶ Watch: First previous bug: CVE-2024-05S2 (UAF due to remapping) (10:10)

The discovery of CVE-2025-2136 and its lineage of related vulnerabilities offers several critical defensive implications for kernel developers, security researchers, and system administrators:

1. Fundamental Re-evaluation of Shared Memory Design: The recurring nature of io_uring memory sharing bugs underscores the inherent complexity of managing shared memory in a high-performance, concurrent kernel environment. Developers must approach these designs with extreme caution, rigorously defining ownership, lifetime, and access rules. This includes io_uring itself, but also other complex subsystems like GPU drivers that heavily rely on shared memory.

1. RCU's Limitations: Modification vs. Freeing: A key lesson from this talk is a nuanced understanding of RCU. While RCU effectively prevents deadlocks and ensures objects are not freed prematurely while readers are active, it does not prevent concurrent modification of an object's fields by a writer. Developers relying on RCU must still implement additional synchronization (e.g., atomic operations, spinlocks) for critical fields that can be modified concurrently by writers, even if readers are present.

1. Vigilance with Reference Counts: The vulnerability's root cause involved a "dummy operation" that forcefully reset a reference count. This highlights the danger of operations that modify reference counts without proper checks or atomicity, especially when other mechanisms (like RCU or other locks) are assumed to provide sufficient protection. All code paths that interact with object reference counts must be meticulously reviewed to ensure they correctly reflect object lifetime and concurrent access patterns.

1. Thorough Testing of "Upgrade" and "Reinitialization" Paths: Operations that "upgrade" or reinitialize existing objects (like converting a provided buffer to a ring buffer) are particularly hazardous. These paths often involve resetting flags and fields, which can lead to unexpected state transitions and race conditions if not handled atomically and with full consideration of concurrent access by other handlers.

1. Enhanced Fuzzing and Static Analysis for io_uring: Given io_uring's history as a vulnerability hotbed, continued and improved fuzzing efforts specifically targeting its memory sharing, object lifecycle management, and concurrent operations are essential. Static analysis tools should also be tuned to detect patterns indicative of race conditions, especially around reference count manipulation and union fields.

1. Kernel Hardening Measures: While KASLR was bypassed in the CTF environment, its continued development and strengthening remain important. Broader kernel hardening strategies, including stricter memory sanitizers (like KASAN), can help detect UAFs earlier in the development and testing cycle.

1. Prompt Patching and Updates: For system administrators, the implication is clear: keeping the Linux kernel updated with the latest security patches, particularly those addressing io_uring vulnerabilities, is paramount. The complexity of these bugs means they are often exploited in the wild shortly after public disclosure.

The fix for CVE-2025-2136 involves modifying the "upgrade" operation to allocate a new io_buffer object instead of reusing the old provided buffer. This prevents the is_mapped flag and reference count of the original object from being updated in a way that creates the race condition, effectively breaking the specific vulnerability chain.

Key Takeaways

• Memory sharing is deceptively complex: Subsystems like io_uring and GPU drivers, which rely heavily on shared memory for performance, are inherently difficult to secure due to complex ownership rules, object lifetime management, and concurrent access patterns.
• RCU protects against freeing, not modification: Read-Copy Update (RCU) is excellent for non-blocking reads and deferred object freeing, but it does not prevent concurrent modification of an object's fields by writers. Developers must use additional synchronization for critical fields.
• Reference count handling is critical: Be extremely cautious with operations that modify object reference counts. Even "dummy" operations that force a refcount value can introduce severe vulnerabilities when combined with concurrency and other reference count updates.
• "Upgrade" and reinitialization paths are high-risk: Code that reuses or "upgrades" an existing object by reinitializing its fields is prone to introducing race conditions, especially if these operations are not fully atomic or don't account for concurrent access.
• io_uring remains a significant attack surface: Its performance-driven design, involving intricate memory management and asynchronous operations, makes it a continuous source of high-impact kernel vulnerabilities.

About the Speaker(s)

Chih-Yen Chang, known as Pumpkin, is a security researcher at DEVCORE. His primary focus areas include the Linux kernel and Android operating system security, where he delves into complex vulnerabilities and exploitation techniques. In addition to his work on operating systems, Pumpkin has also contributed to research concerning virtual machine security. His expertise lies in uncovering deep-seated architectural flaws and race conditions within critical system components, as demonstrated by his detailed analysis of io_uring vulnerabilities.