Decompiling the Synergy: An Empirical Study of Human-LLM Teaming in Software Reverse Engineering

Decompiling the Synergy: An Empirical Study of Human-LLM Teaming in Software Reverse Engineering

Speakers: Zion Leonahenahe Basque

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=I_jM-zA0BLs

Overview

This large-scale empirical study examines how LLMs impact software reverse engineering performance through a controlled experiment with 48 practitioners (24 experts, 24 novices) generating 109 hours of recorded reverse engineering activity and 1,517 LLM queries. The key finding: LLMs provide a 2x improvement in software understanding rate for novices, bringing them to expert-level performance primarily through function summarization. However, experts showed negligible improvement when using LLMs, suggesting that current LLMs complement knowledge gaps rather than accelerating expert workflows.

The study is based on a prior survey of 153 practitioners who identified six distinct LLM applications in reverse engineering, all centered on recovering semantics from decompiled code. The research was conducted across Arizona State University, University of Padova, and Duraccom.

Background

▶ Watch: Reverse engineering as human-dominated, expensive process (0:00)

Binary reverse engineering is a human-dominated, expensive, and expertise-intensive process critical for malware analysis, vulnerability discovery, and program reconstruction. Practitioners use decompilers (like IDA Pro) to translate binaries into higher-level representations, but decompiled code lacks semantic information (function names, variable types, comments). LLMs are increasingly used to recover these semantics, but their actual impact on the reverse engineering process has not been rigorously measured.

Key Findings

▶ Watch: Study design: A/B experiment with 48 practitioners on CTF challenges (4:00)

• Novices gain 2x improvement in software understanding rate with LLM assistance, reaching expert-level performance
• Experts show negligible improvement from LLM use, despite using LLMs the same amount as novices
• Function summarization is the primary beneficial LLM application for novices (e.g., identifying TEA encryption algorithm in decompiled code)
• Both experts and novices use LLMs equally often (~1,517 total queries), contrary to expectations that novices would use them more
• Six distinct LLM applications identified by 153 practitioners: all focused on semantic recovery from decompiled code
• LLMs close the expertise gap by providing knowledge that novices lack (algorithm recognition, pattern identification)

Technical Deep Dive

▶ Watch: Web-based instrumented IDA Pro environment with LLM integration (6:00)

The study used an A/B control/treatment design: each practitioner solved two CTF-inspired binary challenges (RPC server, encrypted filesystem manager; ~900 lines of C, ~30 functions each, compiled and stripped). One challenge used LLM assistance, one did not. A highly instrumented web-based IDA Pro environment captured all interactions. Scoring used "checkpoints" (critical understanding milestones, 8 per challenge), with software understanding rate = correct checkpoints / time. The instrumented platform ran in-browser with full virtual desktop access.

Demo / Proof of Concept

▶ Watch: Key finding: 2x improvement for novices, negligible for experts (10:00)

48 practitioners completed the study, generating 109 hours of recorded reverse engineering across two binary challenges. The platform provided IDA Pro with built-in LLM features (summarize function, identify vulnerability, free prompt) accessible alongside the decompiler.

Defensive Implications

▶ Watch: Function summarization as primary benefit: TEA encryption example (12:00)

For security teams managing reverse engineering workloads: LLMs can meaningfully expand the effective workforce by enabling junior analysts to perform at expert levels for semantic understanding tasks. However, don't expect productivity gains from senior engineers. Training programs should focus on teaching effective LLM collaboration skills (prompt engineering for reverse engineering). Organizations should ensure LLM access for all analysts, not just beginners.

Key Takeaways

• LLMs provide 2x improvement for novice reverse engineers but negligible improvement for experts
• Function summarization is the primary beneficial application, helping novices identify algorithms and patterns they would otherwise miss
• Experts and novices use LLMs equally often, but benefits don't transfer equally
• LLMs close the expertise gap rather than creating universal speedup
• 153-practitioner survey identified six distinct LLM applications, all focused on semantic recovery
• The study provides the first controlled measurement of LLM impact on reverse engineering workflow

About the Speaker(s)

Zion Leonahenahe Basque is the first author from Arizona State University. The research is a collaboration with the University of Padova and Duraccom, combining expertise in binary analysis, human factors research, and software engineering.