Decompiling the Synergy: An Empirical Study of Human-LLM Teaming in Software Reverse Engineering
Zion Leonahenahe Basque
Network and Distributed System Security (NDSS) Symposium 2026 · Day 3 · AI & Web Security
This large-scale empirical study examines how **LLMs impact software reverse engineering** performance through a controlled experiment with **48 practitioners** (24 experts, 24 novices) generating **109 hours of recorded reverse engineering activity** and **1,517 LLM queries**. The key finding: LLMs provide a **2x improvement in software understanding rate for novices**, bringing them to expert-level performance primarily through function summarization. However, **experts showed negligible improvement** when using LLMs, suggesting that current LLMs complement knowledge gaps rather than accelerating expert workflows.
AI review
The first rigorous controlled study of LLM impact on reverse engineering, with 48 practitioners, 109 hours of recorded activity, and 1,517 LLM queries. The finding that novices gain 2x improvement while experts gain nothing is the most important result for anyone building or managing RE teams. The study design (A/B control, checkpoint-based scoring, instrumented IDA Pro) sets a new standard for evaluating LLM-assisted security tools.