(Un-)Natural Language Processing: Defensive AI in Practice
Emanuel Seemann
Swiss Cyber Storm 2024 · Day 1 · Scenario
Overview
In an era where cyber threats evolve at an unprecedented pace, traditional signature-based security mechanisms often struggle to keep up. Emanuel Seemann's talk, "(Un-)Natural Language Processing: Defensive AI in Practice," presented at Swiss Cyber Storm, delves into a novel approach to tackle this challenge: leveraging artificial intelligence, specifically techniques inspired by Natural Language Processing (NLP), to detect and prevent sophisticated attacks. Seemann, an AI specialist at the French startup Crow, outlined his team's research into developing an "AI monkey" capable of identifying anomalous patterns in Uniform Resource Identifiers (URIs), thereby offering a proactive defense against zero-day exploits.

Key moments
- 2:30 The 'Mages' problem: timely updates for novel exploits
- 5:40 Introducing the 'monkey' AI for anomaly detection
- 8:00 Technical approach: transforming URIs into English-like sentences
- 11:30 The critical 'bad data' problem in research
- 18:00 Deep learning trade-off: monkey (good/slow) vs. leopard (fast/stupid)
- 20:20 Crow open-sourcing models and training 'leopards' for defense
- 22:40 Q&A: Combining ML confidence with traditional security rules
(Un-)Natural Language Processing: Defensive AI in Practice
Speakers: Emanuel Seemann, AI Specialist, Crow
Conference: Swiss Cyber Storm
YouTube: https://www.youtube.com/watch?v=rzkNJfTFph4
Overview
In an era where cyber threats evolve at an unprecedented pace, traditional signature-based security mechanisms often struggle to keep up. Emanuel Seemann's talk, "(Un-)Natural Language Processing: Defensive AI in Practice," presented at Swiss Cyber Storm, delves into a novel approach to tackle this challenge: leveraging artificial intelligence, specifically techniques inspired by Natural Language Processing (NLP), to detect and prevent sophisticated attacks. Seemann, an AI specialist at the French startup Crow, outlined his team's research into developing an "AI monkey" capable of identifying anomalous patterns in Uniform Resource Identifiers (URIs), thereby offering a proactive defense against zero-day exploits.
The core problem addressed is the inherent latency and cost associated with manually updating security signatures for every new vulnerability. This talk explores how AI can automate this process, enabling security systems to adapt to novel threats without constant human intervention or costly updates. Seemann emphasizes the critical distinction between theoretical research performance and practical application, highlighting how access to real-world customer data transformed a promising concept into a viable defensive tool.
This research matters significantly for the cybersecurity landscape. It offers a potential paradigm shift from reactive, signature-dependent defense to a more adaptive, intelligent system that can identify previously unseen attacks. By reducing the burden on security vendors and customers alike, and by proposing a robust, generalizable AI solution, Crow's work paves the way for more resilient and efficient network defense strategies, ultimately aiming to open-source these advanced capabilities to the broader community.
Background
▶ Watch: The 'Mages' problem: timely updates for novel exploits (2:30)
The foundation of most existing security software, whether it be an Intrusion Prevention System (IPS), a Web Application Firewall (WAF), or an antivirus, relies on a common principle: matching observed logs or traffic against predefined rule sets or signatures. For web traffic, this often involves complex regular expressions (regex), while antivirus software might use file hashes to identify known malicious binaries. This system works effectively for known threats, where specific patterns or characteristics have been identified and codified into detection rules.
However, this reliance on signatures creates a fundamental challenge known as the "update problem." As new exploits, particularly zero-day vulnerabilities, emerge, security vendors and their customers face a race against time. Vendors must rapidly develop and deploy new signatures, incurring significant development costs and requiring specialized security analysts—dubbed "Mages" by Seemann—to constantly monitor the threat landscape. For customers, this translates into continuous updates to their security stack, which can be disruptive, resource-intensive, and often difficult to manage, especially in high-availability environments. The speaker illustrates this with an analogy of a shoe store's web logs, showing a typical user journey juxtaposed with a suspicious URI containing strings like shinjas de debug and Java long runtime, which clearly deviate from normal behavior but might not match any existing signature.
The inherent limitation of signature-based detection is its reactive nature: it can only defend against what is already known. This leaves a significant window of vulnerability for novel attacks. The project's initial motivation was to address this by developing an AI-driven anomaly detection system—an "AI monkey"—that could automatically identify anything that deviates from normal patterns. Such a system would theoretically adapt to new attacks autonomously, drastically reducing the need for manual signature updates and the associated costs for both vendors and customers. The core questions driving this research were whether such an AI could be developed, whether it could be generalized across diverse customer environments (i.e., transferred), and if it would be cost-effective to run.
Key Findings
▶ Watch: Technical approach: transforming URIs into English-like sentences (8:00)
The research began by exploring established methodologies for applying Natural Language Processing (NLP) to security data, specifically focusing on URI analysis. The team adopted a method called SECToVE, based on a paper by Niosi, which proposes transforming URIs into a format resembling English sentences. This allows the application of sophisticated NLP techniques, which benefit from extensive research in language understanding. The initial pipeline involved converting URIs into embeddings (vector representations) and then classifying these embeddings as benign or malicious using a second machine learning model. Early experiments on research data showed promising results, with the model demonstrating learning capabilities indicated by a decreasing Val Epoch loss reaching as low as 0.04.
However, a critical challenge emerged when these models were applied to real-world scenarios. The speaker highlighted the "data problem" prevalent in security research: publicly available datasets are often synthetic, outdated, or lack the diversity and complexity of actual production logs. This issue is so severe that a recent paper published this year specifically called for a cessation of reliance on these inadequate benchmark datasets for Network Intrusion Detection Systems (NIDS) research. Consequently, the models trained on these suboptimal datasets performed poorly on Crow's internal logs, generating numerous false positives and failing to generalize to truly novel exploits.
This led to a pivotal shift in the project's strategy: instead of relying on public datasets, Crow leveraged its relationships with customers to obtain NDA-protected, real-world log data. This access to high-quality, diverse customer logs proved transformative. By training models on this superior data, the team observed a "significantly better" performance. To validate these new models, they used nuclei templates, an open-source vulnerability scanner, to generate known attacks and compare them against the AI's detections. The results confirmed that these models not only generalized effectively to new attacks but also adapted better to varied URL structures across different customer environments.
A notable refinement during the project was the evolution from an initial anomaly detection objective to a classification problem. While the project was conceived around anomaly detection, practical implementation revealed that such systems, which typically require maintaining and querying a "state" of what constitutes normal behavior, were too slow for real-time security applications. Therefore, the project pivoted to a classification approach, which proved more efficient for the required speed of reaction. Ultimately, the key findings underscored that while researchers provide excellent architectural ideas, the efficacy of these AI models in network defense hinges entirely on the quality and diversity of the training data.
Technical Deep Dive
▶ Watch: The critical 'bad data' problem in research (11:30)
The core technical innovation presented is the transformation of raw URI data into a structured format amenable to advanced NLP techniques, enabling machine learning models to identify malicious patterns. This process, inspired by the SECToVE method, involves several critical steps.
First, URI tokenization is performed. URIs, unlike natural language sentences, have a distinct structure defined by special characters and encoding schemes. The method splits URIs based on URL structure tokens such as / (slash), ? (question mark), and = (equals sign). Additionally, URL encodings, like %20 for a space, are recognized and separated. For instance, a URI like store/search?query=Air%20Jordan would be parsed into tokens such as store, search, query, Air, and Jordan. This effectively converts a semi-structured URI string into a sequence of terms that resembles an "English sentence," albeit one with some grammatical imperfections. The example given in the talk, shinjas de debug for name Java long run time, clearly illustrates how a malicious or anomalous URI segment would stand out as "not very English," making it distinguishable from legitimate requests.
Once the URI is tokenized into this "English sentence" format, it enters a standard NLP pipeline for further processing:
- Tokenizer Step: The sequence of textual tokens is converted into a sequence of numerical representations. Each unique word or token is mapped to a specific integer ID from a vocabulary. This numerical conversion is essential as machine learning models operate on numerical data.
- Embedding Step: This is a crucial and computationally intensive stage. The sequence of numerical IDs is transformed into a vector of fixed dimension, known as an embedding. Embeddings are dense, low-dimensional representations that capture semantic relationships between words. Words with similar meanings or contexts will have similar vector representations. For example,
AirandJordanin the context of shoes might be closer in the embedding space thanJavaandRuntimeif the model is trained on shoe-related queries. This step condenses the complex information of the URI into a compact, numerical format suitable for machine learning. - Classifier Model: The final embedding vector is then fed into a second machine learning model, typically a neural network classifier. This model is trained to distinguish between benign and malicious URI embeddings. During training, the model learns to map specific embedding patterns to "good" or "bad" labels. The speaker noted that models achieved a Val Epoch loss of 0.04, indicating successful learning on research data, although this did not translate directly to real-world efficacy without better data.
The transition from theoretical research to practical application necessitated significant data acquisition efforts. As public datasets proved inadequate, Crow obtained NDA-protected customer logs. To label and validate these real-world logs, the team utilized nuclei templates, an open-source vulnerability scanner. This tool allowed them to generate various known attacks against their test environments, effectively creating a ground truth for training and evaluating the AI models. This practical approach to data generation and labeling was instrumental in overcoming the limitations of synthetic data and enabling the models to generalize effectively to new attacks and diverse customer URL structures.
The proposed implementation strategy also includes a hybrid approach. The AI model, after evaluating a URI, will output a confidence percentage indicating its likelihood of being malicious. This confidence score can then be integrated with existing, traditional rule-based systems (e.g., defined in YAML files within Crow's product). For instance, a high AI confidence score could trigger a more aggressive set of traditional rules (e.g., specific regex patterns for SQL injection) that might otherwise generate too many false positives if applied broadly. This layered defense aims to combine the AI's ability to detect novel anomalies with the precision of established rules, enhancing overall detection accuracy while minimizing disruptive false alarms.
Demo / Proof of Concept
▶ Watch: Crow open-sourcing models and training 'leopards' for defense (20:20)
While the talk did not feature a live demonstration of the AI-driven security system in action, the speaker extensively discussed the methodology and results derived from testing the models on real customer data. The "Proof of Concept" was primarily validated through rigorous internal testing rather than a public demo.
The validation process involved obtaining NDA-protected customer logs, which provided a realistic and diverse dataset. To train and evaluate the models, Crow's team utilized nuclei templates, an open-source vulnerability scanner. This tool allowed them to generate a wide array of known attacks against test environments, which were then used to label data and assess the AI's ability to correctly identify malicious traffic. The success of this internal validation, particularly the models' improved generalization to new attacks and varied URL structures on real data, served as the primary evidence for the system's effectiveness. The speaker's emphasis was on the practical challenges and solutions in developing such a system, rather than a live display of its capabilities.
Defensive Implications
▶ Watch: Q&A: Combining ML confidence with traditional security rules (22:40)
The research presented by Emanuel Seemann carries significant implications for cybersecurity defenders, offering a path towards more resilient and adaptive protection against evolving threats.
Firstly, the development of an AI-driven system capable of identifying anomalous URI patterns provides a first line of defense against novel threats, including zero-day exploits. Unlike traditional signature-based systems that are inherently reactive, this AI approach offers a proactive capability, reducing the window of vulnerability between the emergence of a new attack and the deployment of a corresponding defense. This significantly reduces the operational cost for security vendors, who spend considerable resources on constantly writing new signatures, and for customers, who bear the burden of continuous updates.
Secondly, the talk underscores the paramount importance of real-world, diverse training data. Defenders and researchers should actively seek opportunities to contribute or gain access to anonymized, high-quality log data. The dramatic improvement in model performance observed when moving from synthetic research data to actual customer logs highlights that the effectiveness of AI in security is directly proportional to the quality of its training inputs. This calls for greater collaboration between industry and academia to bridge the "data problem."
Thirdly, the proposed architecture facilitates a hybrid defense strategy. By having the AI model output a confidence percentage for potential maliciousness, defenders can integrate this insight with their existing, fine-tuned rule-based systems. For example, a high AI confidence score could trigger stricter scrutiny by traditional WAF rules or prompt an alert that would otherwise be suppressed due to the risk of false positives. This layered approach allows organizations to leverage the AI's broad detection capabilities without completely abandoning the precision and control offered by established rule sets, thereby reducing false positives and improving the overall signal-to-noise ratio in security alerts.
However, implementing such advanced AI models comes with practical considerations, notably the performance trade-off. Seemann introduced the "Deep learning tradeoff," likening fast, less accurate models to "leopards" and slower, more accurate models to "monkeys." This directly impacts the choice between in-band and out-of-band security deployments. In-band systems (where security checks occur before traffic reaches the application) offer maximum security but can introduce latency, impacting user experience and key metrics like Time to First Byte (TTFB), which Google uses for search ranking. Out-of-band systems (where security checks happen asynchronously) maintain application performance but risk an attacker compromising the server before detection. Defenders must carefully weigh these trade-offs, balancing security posture with user experience and business requirements.
Furthermore, the computational demands of large AI models (the "monkeys") can be substantial, often requiring specialized hardware accelerators like GPUs. This can be a significant cost barrier for many organizations. The talk highlighted ongoing research into optimizations such as quantization (reducing model precision) and distillation (training smaller models to mimic larger ones) to create smaller, faster "leopard" models that are more practical for deployment on standard server infrastructure without dedicated GPUs. These optimizations are crucial for making advanced AI security solutions accessible and cost-effective for a broader range of enterprises.
Finally, Crow's commitment to open-sourcing these models and associated plugins signifies a valuable contribution to the defensive community. This initiative will empower other organizations and researchers to experiment with, adapt, and deploy these advanced AI capabilities, fostering innovation and collaborative defense against cyber threats.
Key Takeaways
- AI for Novel Threat Detection: Artificial intelligence, particularly NLP techniques applied to URIs, holds significant potential as a first line of defense against novel and zero-day attacks, moving beyond the limitations of traditional signature-based security.
- Data Quality is Paramount: The effectiveness of AI models in cybersecurity is critically dependent on the quality and diversity of training data. Real-world, NDA-protected customer logs proved far superior to synthetic research datasets, enabling models to generalize effectively to new attacks and varied environments.
- Practicality Drives Design: The project shifted from an initial anomaly detection objective to a classification problem due to performance constraints, demonstrating that practical deployment considerations often dictate the choice of AI methodology in real-time security systems.
- Performance vs. Accuracy Trade-off: There is an inherent "Deep learning tradeoff" between the speed and computational cost of an AI model and its detection accuracy. Defenders must carefully consider this balance, along with the implications for in-band vs. out-of-band deployment, and explore optimization techniques like quantization and distillation.
- Hybrid Defense is Optimal: The most robust defensive strategy likely involves a hybrid approach, combining the AI's ability to detect anomalous patterns (providing a confidence score) with existing, finely-tuned rule-based systems to enhance detection precision and reduce false positives.
- Open Source Contribution: Crow plans to open-source its AI models and plugins, making these advanced defensive capabilities accessible to the broader cybersecurity community and fostering collaborative innovation.
About the Speaker(s)
Emanuel Seemann is an AI Specialist at Crow, a French startup founded in 2020. Based in Zurich, Seemann studied mathematics and has been with Crow for over two years, making him an "old hat" in the young, remote-first company of 35 people distributed across Europe. His primary role involves anything related to AI within Crow, contributing to their open-source Intrusion Prevention System (IDPS) and Web Application Firewall (WAF). Seemann's work focuses on applying AI to enhance defensive capabilities, as demonstrated in his research on (un-)natural language processing for security.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
Seemann's talk is a refreshingly honest look at the pain of bolting ML onto a WAF. It isn't a revolution in threat detection, but rather a solid engineering retrospective on why public NIDS datasets are garbage and why stateful anomaly detection melts in production. A competent session that strips the marketing paint off AI-powered security.
Heather Calloway (CISO) — SOLID
A rare AI security talk that survives contact with reality. The speaker rightly identifies that academic network datasets are useless for training production models and that latency is the true enemy of AI at the edge.