Let's Talk About the AI Apocalypse

Let's Talk About the AI Apocalypse

Speaker: Dylan Ayrey

Conference: BSidesSF 2025 — April 26-27, 2025, San Francisco

YouTube: Watch on YouTube

Reading time: ~8 minutes

TL;DR

Dylan Ayrey, creator of TruffleHog and CEO of Truffle Security, delivers a hands-on workshop demonstrating that the ingredients for building a self-replicating, ransomware-spreading AI worm already exist — and cost almost nothing to assemble. Using fine-tuned open-source models running on consumer hardware, Ayrey shows that AI does not reinvent the hacking formula, but dramatically lowers the barrier to executing it at scale.

Introduction

The title "Let's Talk About the AI Apocalypse" is accompanied by a deliberate caveat: this talk is non-hyperbolic. Dylan Ayrey, founder of Truffle Security and author of the open-source secret scanning tool TruffleHog, frames the session not as a talk but as a tutorial — a workshop whose goal is to leave every person in the room capable of building "the most destructive, infectious computer worm ever invented in the history of mankind."

That claim earns skepticism, which Ayrey spends the rest of the session methodically dismantling. The argument is not that AI has changed the fundamental structure of offensive security. It has not. The argument is that AI is collapsing the expertise barrier that has kept the most catastrophic attack classes confined to nation-state actors and small numbers of highly skilled attackers.

From Morris to NotPetya: The Formula That Never Changed

▶ Watch: History of computer worms (3:30)

Ayrey begins in 1988 with the Morris Worm, which infected roughly 20% of the connected internet using four techniques:

1. A debug flag in sendmail that allowed remote code execution — still an OWASP Top 10 issue today (security misconfiguration).
2. A buffer overflow exploit — so persistent that the White House Cyber Director was publicly calling for an end to memory-unsafe languages as recently as last year.
3. IP-based authentication bypass — broken authentication, currently OWASP Top 2.
4. Credential harvesting and reuse from infected hosts — still one of the most effective lateral movement techniques in existence.

The formula — pre-exploitation followed by post-exploitation — has not changed in 37 years. What has changed is cost. The Morris Worm was expensive to clean up but generated no revenue. The NotPetya worm of 2017 caused over $10 billion in damage. The trend line, Ayrey shows, points clearly toward a worm that will cost hundreds of billions of dollars. NotPetya introduced the critical accelerant: cryptocurrency-backed ransomware gave attackers a financial incentive to maintain and update their worm, and money has been flowing into the NotPetya wallets ever since 2017.

The Teenage Threat Model and Why AI Changes It

▶ Watch: Teenagers, LLMs, and the threat model (15:00)

The main protection against widespread exploitation, Ayrey argues bluntly, is fear of prosecution — a deterrent that does not apply to countries without extradition treaties and, notably, to teenagers. Many of the most damaging real-world breaches have been perpetrated by minors. If current AI models can code in the 90th percentile, the question is: can a less sophisticated, locally-run model replace the teenage hacker and build a self-propagating worm?

The resource objection is addressed first. Two separate language models run on Ayrey's smartphone in the Android runtime environment — no GPU, no full CPU access. He also ran a model on a compromised smart bed IoT device with compute roughly equivalent to a Raspberry Pi. The primary constraint is storage: the Llama advanced reasoning 8-billion-parameter model requires approximately 15 GB; a 3-billion-parameter model fits in under 5 GB. Compute is not the limiting factor.

How Models Actually Work — and Why Safety Alignment Is Removable

▶ Watch: Model internals and alignment (20:00)

The common misconception that language models are "just next-word predictors trained on garbage" leads to underestimating them, Ayrey argues. The base model is trained on all public internet data — including GitHub, Wikipedia, every hacking guide ever published. Safety behavior is not a property of that base training; it is added afterward through alignment steps, concluded with a safety alignment that produces refusals.

The architecture insight matters: at the foundation of every LLM is an embedding — a vector space where every token is represented by spatial coordinates. The relative positions of those vectors carry meaning. The classic example: plotting vectors for "man," "woman," and "king," then adding the distance and direction between man and woman to king, produces the vector for "queen." The model stores facts not as words but as positions in this space — including the fact of how to attack port 3306, and the separate fact of how to refuse when asked.

Alignment is simply training the model to retrieve the refusal vector instead of the exploit vector. Research has demonstrated that there is a single identifiable refusal vector, and it can be inexpensively removed. Ayrey demonstrated this by using reinforcement learning from human feedback (RLHF) — generating two outputs for the same input via temperature sampling, selecting the less-refusal output, and reinforcing it. Thirty minutes, sitting on his couch, pennies in compute cost, running on a consumer GPU — probably overkill, he notes.

Building the Worm: Fine-Tuning for Code Execution

▶ Watch: Fine-tuning the attack model (32:00)

With an unsafe model in hand, the next challenge is behavioral: the model produced markdown-formatted responses instead of executable code blocks, likely a Meta alignment artifact. Ayrey solved this with an additional fine-tuning pass using synthetic data — hacking guides for multiple CVEs, reformatted to include code blocks and troubleshooting steps (for example, knowing to add -k for self-signed certificate errors). Some data came from Hugging Face, where pre-existing hacking datasets already exist for model training.

The result: given an nmap scan showing SSH exposed, the model correctly identified that direct SSH access would fail, diagnosed that a credential-spraying attack was appropriate, identified Metasploit as the tool, noticed Metasploit was absent from the execution environment, and went to install it. Closer to correct behavior with Elasticsearch CVEs, though not fully there — the model got the format of requests right but generated endpoints from memory rather than the precise CVE documentation, likely because each CVE appeared only once in training data.

The worm architecture uses a Python supervisor — no external dependencies, no ChatGPT API call that could be shut down remotely. The supervisor provides the model with nmap output for a target host, puts it in pre-exploitation or post-exploitation mode, and executes code from model-generated code blocks, feeding the output back into the context window. Post-exploitation, the supervisor runs TruffleHog on the infected host and feeds discovered credentials back to the model.

The Cost and the Conclusion

The fine-tuning cost roughly $20, Ayrey reports. The full threat estimate: a highly capable AI-augmented worm, capable of recognizing and exploiting common CVEs and harvesting credentials from infected systems, built with open-source tools, running on consumer hardware, for approximately the cost of dinner. The asymmetry between offensive and defensive investment is not new — but AI is compressing the required skill level toward near-zero.

Ayrey closes with three concrete defensive priorities: patch faster, minimize credential exposure (TruffleHog remains as relevant as ever), and take memory safety seriously at the language level. The worm formula has not changed since 1988. What is changing is who can execute it.

Notable Quotes

"My goal today is to get every single person in this room capable of walking out and building the most destructive, infectious computer worm ever invented in the history of mankind. This talk is non-hyperbolic."

"The main protection against companies getting hacked is fear of prosecution — which does not apply to countries without extradition, and also does not apply to teenagers."

"The alignment step is simply having the model choose to give us the refusal fact instead of the hacking fact — which is already in its head from training off the entire internet."

Key Takeaways

• The offensive formula is unchanged since 1988. Pre-exploitation and post-exploitation are still the two phases. AI does not reinvent this; it automates it.
• Resource constraints are effectively gone. An 8-billion-parameter model runs on a smartphone, off-GPU. The primary limit is 15 GB of storage.
• Safety alignment is shallow. The facts that enable dangerous behavior are in the base model. The refusal is a single vector that can be removed in 30 minutes for pennies.
• Fine-tuning for offensive behavior costs approximately $20. Synthetic training data with CVE exploitation examples and code block formatting is sufficient to produce a functional attack agent.
• No external dependencies = no kill switch. Worms relying on ChatGPT APIs can be disabled. A self-contained local model cannot. That architectural choice is the critical one.