RTCON: Context-Adaptive Function-Level Fuzzing for RTOS Kernels

RTCON: Context-Adaptive Function-Level Fuzzing for RTOS Kernels

Speakers: Eunkyu Lee

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=ciU-Qqta8qc

Overview

Real-Time Operating System (RTOS) kernels power billions of IoT and embedded devices, providing Bluetooth stacks, Wi-Fi modules, and custom interfaces. Many lack security mitigations like ASLR, making even simple vulnerabilities severely exploitable. This talk presents RTCON, a function-level fuzzing framework that overcomes the key limitation of existing RTOS fuzzers: their inability to reach deeply nested functions where most vulnerabilities reside. RTCON solves this through adaptive context generation (dynamically creating valid function context at runtime), combined with multi-layer classification to distinguish genuine bugs from false positives.

RTCON discovered 27 bugs across 4 major RTOS kernels (Zephyr, RIOT, PrIoT, ThreadX), with 14 CVEs assigned. The multi-layer classification achieves 92.7% precision for high-confidence bugs, dramatically reducing false positive rates compared to naive function-level fuzzing.

Background

▶ Watch: RTOS security challenges: deep vulnerabilities in embedded kernels (0:00)

RTOS kernels provide diverse subsystems (Bluetooth, Wi-Fi, custom interfaces) that expand external attack surfaces. Vulnerabilities typically reside in deeply nested functions that require specific program states to reach. Existing fuzzing techniques achieve high coverage near entry points but suffer dramatic coverage drops deeper in the call graph.

Function-level fuzzing -- directly testing individual functions rather than fuzzing from program entry points -- can reach deep code. However, it introduces two critical challenges: (1) functions require context variables (parameters, global state) that are unavailable when the function is called directly without its parent call chain, leading to immediate crashes or unreachable code; (2) crashes may be invalid because constraints enforced by parent functions are bypassed, generating high false positive rates.

Key Findings

▶ Watch: RTCON overview: context generation, instrumentation, multi-layer classification (4:00)

• 27 bugs discovered across Zephyr, RIOT, PrIoT, and ThreadX RTOS kernels, with 14 CVEs assigned
• 92.7% precision for high-confidence bugs using multi-layer classification, versus only 5.8% for low-confidence bugs
• Significant coverage improvement over state-of-the-art API fuzzers and manual harnesses, particularly in deeply nested functions
• Adaptive context generation effectively handles nested pointer structures, conditional branches with context-dependent values, and complex data types
• Bugs found are deeper than those discoverable by existing approaches, which are limited to bugs near entry points
• The technique is not RTOS-specific and can generalize to general libraries and other embedded software

Technical Deep Dive

▶ Watch: Adaptive context sanitization and value generation hooks (6:00)

RTCON operates in three stages:

Inter-procedural Taint Analysis: Starting from function parameters (excluding user input), taint propagation identifies all context-related variables within each target function. Variables derived from function parameters are tainted; variables derived only from user input are untainted. This distinction determines which variables need synthetic context generation.

Adaptive Context Generation: Two types of instrumentation hooks are inserted:

• Context sanitization hooks: Inserted before pointer dereferences of context variables, these hooks detect null/invalid pointers and allocate valid memory, preventing crashes from uninitialized context. For nested structures (multiple sequential load operations), hooks are inserted at each level.
• Context value generation hooks: Inserted before comparison operations involving context variables, these hooks set the context variable to values close to the comparison operand, maximizing the chance of satisfying branch conditions and reaching deeper code. Candidate values are selected from the operand's neighborhood.

Multi-Layer Classification: Crashes from target function fuzzing are verified by re-fuzzing from a tier-layer function (entry-level handler with no further parent). If the same crash reproduces from the tier-layer function, it is classified as high confidence (92.7% precision). If it only reproduces from the direct function call, it is classified as low confidence (5.8% precision). Tier-layer functions are identified through bidirectional call graph analysis with indirect call resolution.

Demo / Proof of Concept

▶ Watch: Multi-layer classification: 92.7% precision for high-confidence bugs (10:00)

RTCON was evaluated on four major RTOS kernels. Compared to state-of-the-art fuzzers and manually crafted harnesses, RTCON achieves significantly better line coverage, particularly in deeply nested functions. Manual harnesses achieve comparable coverage but require substantial expert effort. The 27 discovered bugs span multiple vulnerability types, with 14 receiving CVE assignments from respective vendors.

Defensive Implications

▶ Watch: Results: 27 bugs, 14 CVEs across Zephyr, RIOT, PrIoT, ThreadX (12:00)

• Embedded device security: RTOS kernels lack security mitigations that desktop/server OSes provide, making vulnerability discovery critical
• Automated deep testing: RTCON eliminates the need for manually crafted test harnesses, enabling automated security testing of RTOS subsystems
• Supply chain verification: Organizations deploying RTOS-based products can use function-level fuzzing to test third-party kernel components
• False positive management: The 92.7% precision for high-confidence bugs makes the tool practical for security teams who cannot afford to triage large numbers of false positives

Key Takeaways

• RTOS vulnerabilities typically reside in deeply nested functions unreachable by conventional fuzzing from entry points
• Adaptive context generation dynamically creates valid function context at runtime, enabling function-level fuzzing without manual harness construction
• Multi-layer classification achieves 92.7% precision for high-confidence bugs by verifying crashes from tier-layer functions
• 27 bugs found across Zephyr, RIOT, PrIoT, and ThreadX with 14 CVEs assigned
• The technique generalizes beyond RTOS kernels to any software with deep call hierarchies

About the Speaker(s)

Eunkyu Lee is a researcher at KAIST. The presentation demonstrated strong understanding of both RTOS internals and fuzzing methodology, with the session chair noting they could "ask questions in lab meeting" -- suggesting active KAIST involvement in the research group.