Security Through Obscurity Is Dead and LLMs Killed It
Brendan Dolan-Gavitt (XBOW)
[un]prompted.au 2026 · Day 2 · Main Hall
Overview
In this provocative talk from [un]prompted.au, Brendan Dolan-Gavitt, a Principal Research Scientist at XBOW, declares that software reverse engineering is now "basically a solved problem" thanks to the advent of Large Language Models (LLMs). Dolan-Gavitt argues that the traditional barriers of complexity and time, which once protected proprietary systems and embedded devices through obscurity, have been shattered. His presentation vividly demonstrates how LLMs can rapidly dissect intricate software and hardware interactions, enabling feats of emulation and algorithm reversal previously deemed intractable for human researchers.

Key moments
- 0:30 Bold claim: Reverse engineering is solved
- 2:15 LLMs build emulators from firmware
- 4:45 Live demo: Emulated printer prints PDF
- 7:15 Exploiting the emulated printer live
- 8:45 Discovering Photoshop's hidden currency detector
- 11:00 LLMs reverse engineer complex algorithms
- 13:00 Adversarial attack makes cat "currency"
- 14:00 Live demo: T-shirt triggers detector
Security Through Obscurity Is Dead and LLMs Killed It
Speakers: Brendan Dolan-Gavitt, Principal Research Scientist, XBOW
Conference: [un]prompted.au
YouTube: https://www.youtube.com/watch?v=8rOY3aD1yxY
Overview
In this provocative talk from [un]prompted.au, Brendan Dolan-Gavitt, a Principal Research Scientist at XBOW, declares that software reverse engineering is now "basically a solved problem" thanks to the advent of Large Language Models (LLMs). Dolan-Gavitt argues that the traditional barriers of complexity and time, which once protected proprietary systems and embedded devices through obscurity, have been shattered. His presentation vividly demonstrates how LLMs can rapidly dissect intricate software and hardware interactions, enabling feats of emulation and algorithm reversal previously deemed intractable for human researchers.
The core message is a stark warning to the security community: relying on the difficulty of reverse engineering as a defensive strategy is no longer viable. LLMs have dramatically lowered the entry barrier for understanding and manipulating complex systems, compressing years or decades of human effort into mere days. This paradigm shift has profound implications for how we approach security in everything from consumer electronics to critical infrastructure, demanding a fundamental re-evaluation of defensive postures and a greater emphasis on robust, provable security rather than mere concealment.
Background
▶ Watch: Bold claim: Reverse engineering is solved (0:30)
For decades, the security of many systems, particularly embedded devices and proprietary software, has implicitly relied on security through obscurity. These systems often feature custom hardware, undocumented interfaces, complex firmware, and intricate algorithms that are costly and time-consuming for humans to reverse engineer. The sheer effort required to understand such systems often outweighed the potential benefits for attackers, leaving many vulnerabilities undiscovered or unexploited.
The challenges were particularly acute in two areas highlighted by Dolan-Gavitt:
- Embedded System Emulation: Building faithful emulators for embedded devices is a notoriously difficult task. These devices run on diverse architectures, interact with unique peripherals, and often lack standard operating systems or debugging interfaces. Automating the process of creating functional emulators from raw firmware and hardware specifications was an unsolved problem for many years, despite significant academic effort. Without emulation, security testing often requires physical hardware, which can be expensive, difficult to acquire, and impossible to scale.
- Proprietary Algorithm Reversal: Many software applications incorporate complex, proprietary algorithms, often for intellectual property protection or security functions. A notable example is the currency detection algorithm found in photocopiers and image editing software like Photoshop. While researchers like Steven Murdoch did foundational work in 2004 on such systems, fully reverse engineering the intricate signal processing pipelines involved remained a monumental task. The common misconception of the "Eurion constellation" as the primary detection mechanism underscores the historical lack of deep understanding due to the complexity and obscurity of these systems.
These challenges meant that the cost (in terms of time, expertise, and resources) of performing deep security analysis on such systems was prohibitively high. Attackers and defenders alike were often deterred, allowing obscurity to serve as a de facto security measure, albeit an unreliable one. The arrival of LLMs, however, has fundamentally altered this cost-benefit analysis, making previously insurmountable reverse engineering tasks suddenly achievable.
Key Findings
▶ Watch: Live demo: Emulated printer prints PDF (4:45)
Brendan Dolan-Gavitt's talk presents several pivotal findings regarding the transformative impact of LLMs on software reverse engineering:
- LLMs Automate Emulator Generation: The most striking finding is the ability of LLMs to generate highly faithful emulators for complex embedded systems. By analyzing firmware and hardware drivers, LLMs can construct functional forks of emulation platforms like QEMU, allowing unmodified firmware to run in a virtual environment. This capability drastically reduces the time and effort required for security testing and vulnerability research on embedded devices, removing the dependency on physical hardware.
- LLMs Deconstruct Proprietary Algorithms: LLMs can effectively decompile and translate complex binary code, even those involving intricate signal processing, into higher-level, human-readable languages like Python. This process, verified through input-output traces from a debugger, transforms opaque algorithms into understandable and manipulable forms, thereby stripping away the protection offered by obscurity.
- Enabling Adversarial Attacks on "Obscure" Systems: Once proprietary algorithms are translated into a high-level language, especially one like Python that can be converted to a differentiable framework like PyTorch, they become susceptible to adversarial machine learning techniques. This allows researchers to craft "imperceptible modifications" to inputs that can fool detection systems, as demonstrated with the currency detection algorithm.
- The Demise of Security Through Obscurity: The overarching finding is that LLMs have effectively killed security through obscurity. What once took years or even decades of specialized human effort can now be accomplished in a matter of days with an LLM. This redefines the landscape of vulnerability discovery and exploitation, accelerating the pace at which complex systems can be understood and compromised.
- Lowered Barrier to Entry: The talk implicitly highlights that the barrier to entry for sophisticated reverse engineering tasks has been significantly lowered. Researchers, even those without deep expertise in specific domains like signal processing, can leverage LLMs to overcome knowledge gaps and accelerate their analysis.
These findings collectively underscore a fundamental shift in the security paradigm, where the internal workings of almost any software or hardware system can now be rapidly exposed and analyzed.
Technical Deep Dive
▶ Watch: Discovering Photoshop's hidden currency detector (8:45)
The technical prowess of LLMs in reverse engineering was showcased through two compelling case studies.
Case Study 1: Embedded Emulation – The Hyper-Realistic Printer
Dolan-Gavitt's first example centered on the automated creation of an emulator for an HP printer he owned around 2013. This wasn't just any printer; it was a particularly complex device, running its main operating system on Windows, while its network services were handled by VxWorks operating system programs that communicated with the Windows kernel through a translation layer within a single Windows DLL. This intricate architecture made it an ideal, previously "unsolved problem" test case for automated emulation.
The LLM-driven process, which the speaker referred to as using "Mythos" initially and then subsequent models, involved feeding the printer's firmware and driver information to the LLMs. The key instruction given to the models was to "not take shortcuts, not to cheat" and to "really implement every peripheral, don't stop until it actually works." This guidance was crucial to ensure a faithful emulation rather than a superficial one.
The resulting emulator was a fork of QEMU, capable of running the printer's unmodified firmware. The level of fidelity achieved was extraordinary:
- Network Connectivity: The emulated printer could be put on a network, allowing print jobs to be sent to its port 9100 JetDirect socket.
- Full Print Pipeline Emulation: The system emulated the entire print pipeline, down to the laser firing sequence that goes off to the engine board.
- Physical Hardware Effects: Beyond basic functionality, the emulator modeled real-world printer behaviors such as "low toner," "paper jams," and "aging hardware that produces weird print artifacts." The low toner simulation, for instance, was based on a PhD thesis written at Purdue modeling print quality degradation.
- Interactive Management: Users could interact with the emulated printer's embedded web server and even telnet to it, mirroring the experience with a physical device.
This high-fidelity emulation proved invaluable for security research. Dolan-Gavitt demonstrated how a known vulnerability in the printer's JPEG 2000 parser could be exploited. By dragging a specially crafted PDF file to the virtual printer and hitting print, the exploit ran on the emulated firmware, culminating in a custom logo being drawn on the virtual screen. This capability directly translated to other critical systems, such as an infotainment system running on older GM cars, for which an emulator was assembled in just a "few days," opening up potential attack vectors for other parts of the vehicle.
Case Study 2: Currency Detection Algorithm Reversal – The Counterfeit Cat
The second case study tackled the challenge of reversing a proprietary currency detection algorithm, specifically the one embedded in Photoshop 2026 (and commonly found in color photocopiers). Historically, this algorithm was a black box, protected by its complexity and the specialized knowledge required for signal processing. Dolan-Gavitt admitted that, as a human, the sheer volume and nature of the code ("oh my god, that's a lot of code," "oh my god, I know nothing about signal processing") had deterred him for years.
LLMs, however, provided a solution. The approach involved a systematic, function-by-function porting process:
- Code Translation: Every single function in the compiled binary (or its decompiled C equivalent) was fed to an LLM, with the instruction to port it to Python.
- Semantic Verification: To ensure accuracy, input-output traces collected from a debugger running the original binary were used to verify that the Python code was semantically identical to the original C code. This rigorous verification step was crucial to maintain fidelity.
- High-Level Understanding: This process yielded a "nicer, more compact description" of the algorithm, which the LLM could then summarize into a conceptual diagram. The algorithm was revealed to be an "older school pre-deep learning pipeline" focused on extracting an embedded watermark.
Once the algorithm was in Python, the real power of LLMs combined with modern machine learning techniques emerged.
- Differentiable Conversion: The Python code was converted into PyTorch, a deep learning framework. Crucially, "a couple of the operations" were replaced with differentiable versions, making the entire currency detector differentiable end-to-end.
- Adversarial Attack Generation: With a differentiable model, adversarial attacks could be performed. Dolan-Gavitt utilized projected gradient descent (PGD), a technique from adversarial example literature, to ask: "What is the smallest change to this image that will make it detected as currency?"
- Constraint and Pruning: The attack was constrained to use whole pixel values, and to account for any discrepancies between the smooth differentiable version and the real, non-differentiable oracle, examples were "pruned out by testing them against the real oracle."
The demonstration was striking: an "extremely handsome cat" image, when subjected to these imperceptible modifications, was then flagged by Photoshop as currency. The ultimate proof of concept was a live demo where a volunteer, Mark Dowd, wore a T-shirt adorned with a pattern designed using this adversarial technique. A photo taken of the T-shirt was then opened in Photoshop, which promptly displayed the "No, no, you're not allowed to open that" warning, confirming the successful bypass of the detection system. This entire process effectively dismantled the "obscurity" that had protected the algorithm for two decades.
Demo / Proof of Concept
▶ Watch: LLMs reverse engineer complex algorithms (11:00)
The talk featured two highly effective and entertaining live demonstrations that underscored the core message.
Printer Emulation Demo
The first demonstration brought the hyper-realistic printer emulator to life. The audience was shown a 3D model of the printer on screen, featuring its real, emulated display running the actual firmware.
- Standard Printing: A PDF was sent to the emulated printer's port 9100 JetDirect socket. The display showed the printer processing the job from "tray 2," loaded with "letter paper," culminating in the successful "printing" of the document (converted to a PNG, humorously noted as "perhaps the least efficient way ever devised to convert a PDF to a PNG").
- Sensor Simulation: Dolan-Gavitt then manipulated virtual sensors, demonstrating how opening and closing a virtual printer door would trigger the corresponding "No, no, you need to close that door" message on the emulated display.
- Low Toner Degradation: To illustrate the depth of emulation, he activated the "low toner" simulation. As a result, subsequent print jobs produced "really crappy images," faithfully degrading print quality based on a Purdue PhD thesis on print quality and toner levels.
- Web and Telnet Access: He navigated to the printer's embedded web server, tunneled through an iframe, showcasing the ability to manage the device virtually. He also mentioned telnet access.
- Exploit Execution: The climax of the printer demo involved exploiting a vulnerability. A specially crafted PDF file designed to trigger an exploit in the printer's JPEG 2000 parser was dragged to the emulator. Upon "printing," the exploit successfully ran on the emulated firmware, causing a custom logo to be drawn directly onto the virtual printer's display, clearly demonstrating the utility of such faithful emulation for vulnerability research.
Currency Detection Evasion Demo
The second demonstration focused on the currency detection algorithm.
- Initial Detection: Dolan-Gavitt first showed a picture of Australian currency. When attempting to open it in Photoshop 2026, the software immediately blocked the action, displaying a message about the illegality of counterfeiting and redirecting to an informational website.
- Adversarial Cat: Next, he presented an image of an "extremely handsome cat." After applying an "imperceptible modification" generated by the LLM-derived adversarial attack technique, he attempted to open this modified cat image in Photoshop. To the audience's applause, Photoshop again triggered the currency detection warning, demonstrating that a non-currency image could be made to appear as currency to the algorithm.
- Live T-shirt Proof of Concept: The most ambitious part was the live demonstration with a volunteer, Mark Dowd. Mark wore a specially designed T-shirt featuring a pattern that incorporated the adversarial modifications. Dolan-Gavitt took several pictures of Mark and his T-shirt using his phone, then AirDropped them to his laptop. Upon opening one of these photos in Photoshop, the software again displayed the currency detection warning, proving that the adversarial pattern worked even in a real-world scenario with a photograph, not just a digitally altered image. This concluded the live demo, solidifying the claim that obscurity in such algorithms is effectively dead.
Defensive Implications
▶ Watch: Live demo: T-shirt triggers detector (14:00)
The demise of security through obscurity, as demonstrated by the power of LLMs, carries profound defensive implications for organizations and security professionals:
- Shift from Obscurity to Robustness: Defenders can no longer rely on the complexity, proprietary nature, or sheer volume of their code/hardware as a deterrent. Any undocumented interface, custom peripheral, or complex algorithm should be considered fully reverse-engineered by an attacker leveraging LLMs. The focus must shift from hiding vulnerabilities to building inherently robust and provably secure systems. This means prioritizing secure design principles, thorough threat modeling, and formal verification methods.
- Accelerated Vulnerability Discovery: The speed at which LLMs can dissect complex systems means that the window between a vulnerability existing and its potential discovery by attackers has drastically shrunk. Hardware manufacturers and software vendors must anticipate that their proprietary protocols, undocumented registers, and custom firmware can be rapidly understood. This necessitates a more aggressive and proactive approach to internal security auditing and vulnerability research, potentially using LLM-assisted tools themselves to find flaws before malicious actors do.
- New Attack Vectors for Adversarial ML: The ability to convert proprietary algorithms into differentiable models opens up new avenues for adversarial machine learning attacks. Systems relying on image recognition, audio processing, or other complex data analysis (e.g., malware detection, spam filters, biometric authentication) are now vulnerable to subtle, imperceptible manipulations that can bypass their intended functions. Defenders must integrate adversarial robustness training and deploy robustness evaluation frameworks to protect against these emerging threats.
- Embedded Device Security is Critical: Given the ease of creating faithful emulators, the security posture of embedded devices (from IoT to automotive systems) becomes even more critical. Attackers can now develop and test exploits without needing expensive or hard-to-acquire physical hardware, making large-scale exploitation more feasible. Manufacturers must invest heavily in secure boot, firmware integrity checks, memory protections, and secure update mechanisms for their embedded products.
- Supply Chain Security: The ability to rapidly reverse engineer components means that vulnerabilities introduced anywhere in the supply chain (e.g., in third-party firmware, drivers, or hardware IP) are more easily discoverable. Organizations need enhanced supply chain security audits and component verification to ensure that all elements of their systems meet stringent security standards, assuming full transparency.
- "Dusting Off Old Projects" for Defense: Just as attackers can leverage LLMs to revisit previously too-hard targets, defenders can use this technology to analyze their legacy systems, identify long-standing vulnerabilities, and improve their security posture. This includes using LLMs to understand complex, poorly documented internal systems or to generate comprehensive documentation where it's lacking.
In essence, the age of LLMs demands a defensive strategy rooted in the assumption of full transparency. If a system's inner workings can be revealed in days, its security must be built on intrinsic resilience, not hidden complexity.
Key Takeaways
- Security Through Obscurity Is Dead: LLMs have fundamentally dismantled the protective barrier of obscurity. Complex, proprietary systems and embedded devices that once took years or decades for human reverse engineers to understand can now be analyzed and exploited in a matter of days.
- Automated Emulator Generation: LLMs can generate highly faithful emulators for embedded systems (e.g., QEMU forks) directly from firmware and driver analysis. This capability drastically reduces the cost and time of security testing and exploit development for hardware-dependent targets.
- Proprietary Algorithm Dissection: LLMs can accurately translate complex binary code, including intricate signal processing algorithms, into high-level languages like Python. This enables deeper analysis and the conversion of these algorithms into differentiable models, opening them to adversarial attacks.
- New Era of Adversarial Attacks: The ability to make proprietary algorithms differentiable allows for the creation of imperceptible adversarial examples. These can bypass detection mechanisms (like currency detectors) or manipulate system behavior, posing new threats to various AI/ML-driven security controls.
- Urgent Call for Robust Security: Defenders must abandon reliance on hidden complexity and instead prioritize fundamentally robust, provably secure designs. This includes proactive vulnerability research, adversarial robustness training, and comprehensive secure development lifecycles, assuming full transparency of their systems.
- Empowerment for Security Research: LLMs empower both offensive and defensive security researchers by lowering the barrier to entry for complex tasks. This acceleration of analysis can be leveraged to find and fix vulnerabilities faster, provided organizations embrace these new tools.
About the Speaker(s)
Brendan Dolan-Gavitt is a Principal Research Scientist at XBOW. His work focuses on challenging problems in software and system security. Notably, he spent over five years in academia researching the automation of embedded system emulation, a problem he admits he "got almost nowhere" with until the advent of large language models. He is known for his dedication to pushing the boundaries of what's possible with security research, often taking projects "too far" in pursuit of a complete understanding, such as creating the "most faithfully emulated printer in the world" and even porting Doom to a CPU cooler. His current role at XBOW continues to explore the cutting edge of security analysis and automation.
Reviews
Dr. Zero (Offensive Security Researcher) — MUST SEE
Brendan Dolan-Gavitt actually did the impossible: he made an LLM security talk that isn't complete bullshit. Instead of building another useless natural language wrapper, he used models to automate lifting undocumented binaries into differentiable PyTorch graphs and generating QEMU machine models that actually boot. This is a terrifyingly practical leap in automated reverse engineering that changes the economics of firmware analysis.
Heather Calloway (CISO) — MUST SEE
Brendan Dolan-Gavitt just proved that the economic barrier to reverse-engineering embedded firmware has collapsed. By using LLMs to automatically generate functional digital twins of physical hardware, attackers no longer need the actual devices or months of lab time to build exploits. This fundamentally breaks how we calculate third-party hardware risk and accelerates the threat timeline for our entire appliance footprint.