Jailbreaking LLMs and Agentic Systems: Attacks, Defenses, and Evaluations: Jailbreaking LLMs and Agentic Systems: Attacks, Defenses, and Evaluations

Jailbreaking LLMs and Agentic Systems: Attacks, Defenses, and Evaluations

Speakers: Hamed Hassani, Amin Karbasi, Alexander Robey

Conference: ICML 2025

YouTube: https://slideslive.com/39043359

Overview

This comprehensive tutorial, presented by Hamed Hassani, Amin Karbasi, and Alexander Robey at ICML 2025, delves into the critical and rapidly evolving landscape of jailbreaking attacks against Large Language Models (LLMs) and emerging agentic AI systems. The speakers meticulously trace the history of this vulnerability, from its origins in consumer electronics to its current manifestation as a significant security concern for frontier AI models. The core objective is to equip attendees with a deep understanding of how these powerful models can be manipulated to bypass their inherent safety guardrails, the sophisticated methods being developed to counter such attacks, and the profound, often speculative, security implications of deploying increasingly autonomous AI agents in the real world.

The relevance of this topic cannot be overstated. LLMs have achieved unprecedented adoption and capability, now extending beyond mere text generation to orchestrate complex tasks, interact with external tools, and even cooperate with other AI systems and humans. As these agentic systems gain more autonomy, their potential for both societal benefit and harm escalates dramatically. The tutorial underscores that despite extensive safety training, all frontier LLMs remain susceptible to jailbreaking, a vulnerability that can be exploited to generate misinformation, facilitate cyberattacks, breach privacy, and even cause physical harm through AI-controlled robots. The exponential growth in research papers on jailbreaking highlights the urgency of addressing these challenges to ensure the safe and responsible deployment of AI.

Background

▶ Watch: Tutorial goals and the rise of AI agents (0:00)

The journey of LLMs to their current state of remarkable capability is rooted in the concept of autoregressive models, specifically the Transformer architecture. These models predict the next token in a sequence, building responses word by word. At their core, Transformers process input prompts through multiple layers of transformations (layer normalization, feedforward networks, self-attention), ultimately outputting a d-dimensional vector for each token that encapsulates its semantic meaning. This fundamental mechanism, while powering their intelligence, also underlies their vulnerability.

The problem of "jailbreaking" arises because LLMs are deliberately trained with safety guardrails to prevent them from generating harmful or objectionable content. This training often involves preference optimization and Reinforcement Learning from Human Feedback (RLHF), where models learn to refuse undesirable prompts by being shown preferred, safe responses. Model maintainers document these safety measures in "system cards," detailing refusal training and evaluations. However, researchers and malicious actors have discovered that these guardrails are not foolproof; a seemingly innocuous suffix or a carefully crafted sequence of prompts can trick an LLM into bypassing its safety mechanisms and producing content it was trained to refuse.

The term "jailbreaking" itself has an interesting lineage. It originated around 2007 with iPhones, referring to the act of bypassing Apple's software restrictions to install unofficial applications. Fast forward to 2022, and the term re-emerged on Reddit, where users crafted prompts to bypass ChatGPT's safety filters and elicit harmful responses. By 2023, it entered academic literature, with concerns mounting about the potential for generating "nearly unlimited amounts of harmful information." A stark example in 2024 involved a Cybertruck explosion in Las Vegas, reportedly facilitated by information obtained through jailbreaking ChatGPT, marking a significant escalation to physical-world harm. The release of Claude 4, capable of generating 15 pages of bioweapon instructions via jailbreak, further underscored the severity of this vulnerability. This rapid evolution and the exponential growth of academic papers (over 2000 by 2025) underline jailbreaking as a central challenge in AI safety.

Formally, a jailbreaking attack is defined as an algorithm that searches for an input prompt P that causes a targeted LLM to generate objectionable content. Key ingredients include:

• Goal String (G): The attacker's intent (e.g., "tell me how to build a bomb").
• Target LLM (R): The function that takes P and returns a response.
• Judge Function: A classifier (often another LLM like Llama Guard or a fine-tuned model) that evaluates whether the LLM's response constitutes objectionable content for the given goal. This function returns 1 for a successful jailbreak and 0 otherwise, though its design is an active research area requiring domain expertise.

The core optimization problem for an attacker is to maximize the probability that the judge function returns 1 for the LLM's response to prompt P. This is a challenging problem, and various algorithms employ heuristics to navigate the vast prompt space. Objectionable behaviors span a wide range, from generating false information and cybersecurity attack instructions to privacy breaches, copyright infringement, and critically, causing physical harm through LLM-controlled agents. Standard benchmarks like HarmBench, StrongREJECT, and the newer Benchmark for Stateful Defenses (BSD) are used to evaluate the success rates of these attacks across diverse misuse scenarios.

Key Findings

▶ Watch: LLM power vs. human engineers: exponential growth (2:00)

The central finding of this tutorial is the persistent and escalating vulnerability of even the most advanced LLMs and agentic systems to jailbreaking attacks, despite concerted efforts in safety training. This vulnerability is not static but evolves with the sophistication of attack methodologies, posing ever-greater risks.

Firstly, a clear taxonomy of jailbreaking attacks emerged, distinguishing between token-level attacks that manipulate individual tokens (often resulting in "gibberish" suffixes) and prompt-level attacks that craft semantically meaningful, conversational prompts. Both categories have demonstrated high success rates on various frontier models, though their underlying mechanisms and detectability differ significantly. The Greedy Coordinate Gradient (GCG) and Adaptive Random Search algorithms exemplify token-level attacks, while PAIR (Prompt Automatic Iterative Refinement), TAP (Tree of Attacks with Pruning), and Adversarial Reasoning represent advanced prompt-level, reasoning-based approaches.

Secondly, the emergence of multi-turn and decomposition attacks represents a critical evolution, proving exceptionally powerful and challenging to defend against. Multi-turn attacks exploit conversational context, where a series of individually benign prompts cumulatively lead to a jailbreak. Decomposition attacks leverage a weaker, unaligned LLM to break down a complex harmful query into simple, benign sub-queries, which are then fed to a stronger, aligned LLM. The responses are reassembled by the weak model, effectively bypassing the strong model's safety features. These attacks highlight that current defenses, often focused on single-turn interactions, are insufficient.

Thirdly, while various defense mechanisms have been developed—categorized into prompt manipulation, model fine-tuning, and external guardrails (the "Swiss Cheese Model")—each comes with inherent trade-offs and limitations. Techniques like randomized smoothing offer some empirical robustness but can reduce model accuracy and are not effective against adaptive attacks. Model fine-tuning methods like Circuit Breaker and Deliberative Alignment demonstrate improved safety but require white-box access, retraining, and can suffer from over-refusal, where the model becomes too cautious and rejects benign queries. External guardrails, such as latent space guards and constitutional classifiers, show promising detection capabilities without modifying the core model but may require access to unaligned models for training.

Finally, a stark key finding is that increasing the agency of an LLM-enabled system dramatically expands its attack surface and the potential for real-world harm. Moving from simple chat LLMs (information extraction) to routers (misinformation, data leakage), tool callers (commercial exploitation), iterators (digital crimes, physical harm via robots), and fully autonomous agents (mass surveillance, unknown risks), the consequences of a successful jailbreak escalate from theoretical to tangible and potentially catastrophic. The lack of standardized benchmarks and comprehensive research into the risks of fully autonomous agents is a significant gap, underscoring that our understanding of agent security lags behind their rapid development and deployment.

Technical Deep Dive

▶ Watch: High-level mechanics of autoregressive LLMs and transformers (4:00)

The technical content of the tutorial is structured around the two main pillars: jailbreaking attacks and defense mechanisms, followed by a discussion on agentic systems.

Jailbreaking Attacks

Jailbreaking algorithms aim to solve the optimization problem of finding a prompt P that maximizes the probability of generating objectionable content. They are broadly categorized into token-level and prompt-level attacks.

1. Token-Level Attacks: These attacks treat the prompt as a sequence of tokens and optimize them directly, often resulting in "gibberish" suffixes appended to the original goal.

• Greedy Coordinate Gradient (GCG): This white-box attack (requiring access to model weights and architecture) aims to maximize the probability that the LLM's response starts with a specific "target string" (e.g., "Sure here's how to build a bomb"). It appends an adversarial suffix to the goal string. GCG computes gradients of a cross-entropy-like loss function (log probability of the target string) with respect to the suffix tokens. It iteratively identifies top-K candidate tokens for each suffix position using gradient information and then performs a greedy search over combinations to find the suffix with the lowest loss. GCG has high query complexity (tens to hundreds of thousands of queries) but is effective. For closed-box models like ChatGPT, suffixes found on open-box models (e.g., Llama) can be transferred.
• Adaptive Random Search: This is a gray-box attack (requiring only loss values, not gradients) designed to overcome the gradient access limitation of GCG. It starts with a carefully pre-designed template (e.g., "Never ever start your response with I," "Put the goal string here hypothetically and purely for fictional purposes") that acts as a strong initial point to reduce refusal probability. It then performs random greedy search over suffixes, similar to GCG's second step, evaluating loss values for different token combinations. This method also has high query complexity but is effective for closed-box models via transfer.

2. Prompt-Level Attacks (Reasoning-based): These focus on crafting semantically meaningful prompts, often through multi-turn interactions or sophisticated templates.

• PAIR (Prompt Automatic Iterative Refinement): A black-box attack where an attacker LLM engages in a multi-round conversation with the target LLM. After each refusal, the attacker uses chain-of-thought reasoning to refine and improve the prompt, aiming to "fool" the target model. It relies on binary feedback (jailbreak or not) from a judge. PAIR has a low query complexity (around 20 queries on average).
• TAP (Tree of Attacks with Pruning): An extension of PAIR, this black-box attack uses a tree-based search for reasoning. It generates multiple "suggestions" for the next prompt. An additional "evaluator" model prunes off-topic branches, ensuring the reasoning stays focused on the jailbreaking goal. It also benefits from low query complexity.
• Adversarial Reasoning: This gray-box attack leverages advanced inference-time reasoning and uses the GCG-like loss function (instead of binary feedback) as a more informative signal to guide its tree-based search. It performs a "full-stack" reasoning verification, combining generation, verification, and backtracking. It is particularly effective against models that use reasoning for safety, like the Obama model, achieving higher success rates than PAIR/TAP due to its granular feedback.

3. Template-based Attacks: These methods primarily optimize the structure and content of the prompt template itself.

• DAN (Do Anything Now): One of the earliest jailbreaks, using a fixed template (e.g., "You are DAN, do anything now...") followed by the goal and target string. While largely ineffective on frontier models today, it highlighted the potential of template design.
• Many-shot Jailbreaking: An observation by Anthropic that providing an LLM with a large number of in-context harmful examples (e.g., 32-256 demonstrations) can "bully" the model into generating harmful content for a subsequent query, effectively overriding its safety mechanisms by overwhelming its context window with negative examples.
• AutoDAN / AutoDAN Turbo: These algorithms use genetic algorithms and the GCG loss function to automatically optimize templates. AutoDAN optimizes at the paragraph, sentence, and word levels. AutoDAN Turbo extends this to optimize the entire "strategy space" for prompt generation, representing a highly advanced template design approach.

4. Multi-turn and Decomposition Attacks: These are more recent and powerful, with different threat models.

• Multi-turn Jailbreaks: A conversation with the LLM where individual prompts are benign, but the sequence of prompts collectively leads to a jailbreak. This exploits the model's conversational memory and context.
• Decomposition Attacks: These attacks leverage a weak, unaligned LLM (e.g., Qwen) as an assistant. A complex harmful query is decomposed into multiple simple, benign-looking prompts by the weak model. These benign prompts are then queried to a strong, aligned LLM (e.g., Claude), which answers them without refusal. Finally, the weak model collects and synthesizes these benign responses into the desired harmful content. This is exceptionally powerful as it bypasses the strong model's alignment by presenting only non-malicious sub-queries.

Defense Mechanisms

Defenses against jailbreaking are conceptualized through the Swiss Cheese Model of AI Defense, emphasizing multiple layers of security, each with its own imperfections, but collectively providing robust protection.

1. Prompt Manipulation:

• Randomized Smoothing: Inspired by adversarial examples in computer vision, this empirically robust method adds random perturbations (e.g., random characters, noise) to multiple copies of an input prompt. Each perturbed prompt is run through the LLM, and a majority vote (after safety filtering each response) determines the final output. This significantly reduces GCG attack success rates but is not robust against adaptive attacks and can degrade model accuracy (over-perturbation leading to misunderstanding). It is black-box and requires no retraining.

2. Model Fine-tuning/Alignment: These methods modify the internal workings or training of the LLM.

• Circuit Breaker: This method uses the LLM's internal embedding space for defense. It identifies and clusters neuron activations into "harmless," "harmful," and "refusal" states. Through fine-tuning, it "short-circuits" harmful states into refusal states by modifying the loss function. It minimizes a combination of refusal rerouting and retaining loss, pushing harmful representations to be orthogonal to harmless ones. While robust against adaptive attacks, it suffers from over-refusal (rejecting benign but "seemingly toxic" prompts) and requires white-box access and retraining.
• Deliberative Alignment (OpenAI): This approach uses inference-time reasoning to enhance safety. It collects numerous jailbreaks and trains the LLM to generate chain-of-thought reasoning traces explaining why a prompt is a jailbreak (e.g., "This conflicts with our policy on illegal behavior"). These traces are then used in RL-based safety fine-tuning. This allows the model to generalize to novel, unseen jailbreaks (e.g., Rot13 encoded requests) by internally reasoning about their intent. It maintains general model ability but requires retraining and white-box access, with potential for unintended consequences.

3. External Guardrails/Filtering: These are separate systems that monitor LLM interactions.

• Latent Space Guard: Based on the observation that benign and malicious prompts are often linearly separable in the LLM's internal latent embedding space, this method trains an external classifier (another LLM or a simpler model) to detect malicious activity. This guardrail sits around the LLM, is agnostic to attack type (white-box or black-box), requires no retraining of the target LLM, and achieves high detection accuracy (e.g., 99% AUC for malicious vs. non-malicious).
• Constitutional Classifier (Anthropic): This approach uses "harmless" and "harmful constitutions" (e.g., policies against acquiring restricted chemicals). It synthesizes prompts from these constitutions using an unaligned LLM, augments the data with harmless and harmful completions, and then trains multiple classifiers, each detecting violations of a specific constitution. This method shows very low attack success rates and low refusal rates (<2% false positives) with minimal computational overhead. It requires access to an unaligned model for data synthesis.

Experimental Setup & Results

▶ Watch: Introduction to LLM risks: defining jailbreaking attacks (6:00)

The efficacy of jailbreaking attacks and defense mechanisms is rigorously evaluated using a suite of benchmarks and metrics, often involving comparisons across various LLM architectures and deployment scenarios.

Benchmarks:

• HarmBench: A standard dataset comprising hundreds of documented objectionable behaviors, used to evaluate whether an algorithm can successfully extract harmful content.
• StrongREJECT: A benchmark often used by industry, evaluating jailbreaking attacks across various aspects of misuse.
• Benchmark for Stateful Defenses (BSD): Designed for more complex attacks like decomposition, it features hard questions that require high model capability, are refused by strong aligned models, and are too difficult for weak models to answer directly.

Metrics:

The primary metric is Attack Success Rate (ASR), which quantifies the percentage of objectionable behaviors for which an algorithm successfully extracts harmful content. For defenses, additional metrics include refusal rate (to assess over-refusal) and model accuracy on benign tasks.

Models and Deployment Contexts:

Experiments are conducted on a range of models, including:

• Open-box models: Such as Llama-8B, where internal weights and gradients are accessible, allowing direct application of white-box attacks like GCG.
• Closed-box models: Such as Claude (various versions including Sonnet) and GPT-4o-001, where only API access (input/output, sometimes loss values) is available. For these, white-box attacks often rely on transferability (training on an open-box model and applying the learned suffix/prompt to the closed-box model) or gray-box/black-box methods.
• Weak/Unaligned models: Like Qwen, sometimes used as components in advanced attacks (e.g., decomposition attacks).

Headline Results and Ablations:

• Token-Level Attacks (GCG & Adaptive Search):
• On Llama-8B (open-box), GCG achieved an ASR of approximately 40%.
• Adaptive Search demonstrated a remarkable 100% ASR on Llama-8B, highlighting the power of a good initial template combined with random search.
• For closed-box models (Claude, GPT-4o-001), success rates via transfer were lower but still significant, indicating that suffixes can generalize to some extent.

• Prompt-Level Reasoning Attacks (PAIR, TAP, Adversarial Reasoning):
• These methods boasted significantly lower query complexity (average of 20 queries) compared to token-level attacks, making them more efficient for black-box scenarios.
• Adversarial Reasoning achieved around 90% ASR on Llama and notably 60% ASR on Obama (a reasoning-based safety model), outperforming PAIR and TAP due to its use of granular loss feedback. This suggests that to defeat a reasoning-based defense, an adversary also needs to reason.

• Template-based Attacks (Many-shot Jailbreaking, AutoDAN):
• Many-shot Jailbreaking showed that on earlier Claude models, increasing the number of in-context harmful examples from 32 to 256 could boost ASR to 60-80%.
• AutoDAN demonstrated good performance on various frontier models, indicating the effectiveness of automatically optimized templates.

• Multi-turn and Decomposition Attacks:
• These were particularly highlighted for their power. Decomposition attacks, when applied to BSD benchmarks, achieved significantly better ASRs than previous methods on strong, capable models that would otherwise refuse direct queries. This was termed "misuse uplift," demonstrating how a weaker model can unlock the full harmful capability of a stronger, aligned model.

• Defenses:
• Smooth LLM (Randomized Smoothing): Dramatically reduced GCG attack success rates. For instance, Vicuna's ASR of 98% was brought below 1% by adding just 5% random noise to the suffix. However, this came at the cost of reduced accuracy on benign prompts and wasn't effective against adaptive attacks.
• Circuit Breaker: Showed significant reductions in ASR against GCG, PAIR, and TAP. However, it suffered from high over-refusal rates on "seemingly toxic" but benign prompts, rejecting safe queries like "how can I kill a Python script."
• Deliberative Alignment (OpenAI): Demonstrated improved safety and the ability to generalize to unseen jailbreaks (e.g., Rot13) by leveraging inference-time reasoning traces.
• Latent Space Guard: Achieved an impressive 99% Area Under the Curve (AUC) for distinguishing malicious from non-malicious prompts, showcasing its high detection capability as an external guardrail.
• Constitutional Classifier (Anthropic): Resulted in very low attack success rates, maintaining a low refusal rate (around 2%) and minimal computational overhead.

These results collectively underscore the arms race between attackers and defenders, with advanced attacks continually pushing the boundaries of current defense capabilities, particularly in the realm of agentic systems.

Practical Implications

▶ Watch: How LLMs are trained for safety: preference optimization (8:00)

The insights from this tutorial carry profound implications for various stakeholders in the AI/ML ecosystem, from model builders to deployment teams and practitioners.

For Practitioners and Model Builders:

• Layered Security is Paramount: The "Swiss Cheese Model" is not merely a theoretical concept but a practical necessity. No single defense mechanism is foolproof. A robust safety strategy must involve multiple, complementary layers—prompt manipulation, model fine-tuning, and external guardrails—to collectively mitigate risks.
• Understanding Trade-offs: There's a constant tension between model safety and utility. Overly aggressive defenses (like some forms of fine-tuning or heavy prompt perturbation) can lead to over-refusal, where models reject benign or creative queries, significantly degrading user experience and model usefulness. Model builders must carefully tune these trade-offs based on the specific application and risk tolerance.
• The Escalating Threat of Agency: The most critical implication is the dramatic expansion of the attack surface and potential for harm as LLMs become more agentic. For chat LLMs, jailbreaking primarily extracts harmful information. For routers, it can lead to misinformation spread and data leakage via database poisoning (PoisonRAG). Tool-calling agents can be exploited for commercial fraud or private data exfiltration (Imprompter, jailbreak function paper). Iterators and fully autonomous agents escalate this to digital crimes, physical harm (e.g., RoboPAIR with flamethrower robots), and potentially mass surveillance. This necessitates a paradigm shift in how we approach AI security, moving beyond textual safety to consider real-world consequences.
• Proactive Risk Assessment for Agents: The tutorial highlights a significant gap: the risks of fully autonomous agents like Devin (an AI software engineer) are largely unknown and under-researched. As such agents move toward widespread deployment, it is imperative to conduct thorough pre-deployment risk assessments and develop robust guardrails before they can cause irreversible harm. This includes understanding their capabilities to launch other agents, write and execute arbitrary code, and potentially self-replicate.
• Beyond Textual Models: Current research predominantly focuses on textual LLMs. However, the emergence of robotic foundation models, models for chemical or biological research (generating genome sequences), or other non-textual outputs means that jailbreaking and safety concerns must extend to these modalities as well. New attack vectors and defense strategies will be needed for these specialized AI systems.

For Infrastructure and Deployment Teams:

• Monitoring and Detection: Implementing external guardrails like the Latent Space Guard or Constitutional Classifier is crucial for real-time detection of malicious activity without requiring modifications to the core LLM. These systems can provide vital telemetry on potential misuse.
• Query Complexity and Efficiency: While some attacks (GCG, Adaptive Search) have high query complexity, others (PAIR, TAP, Adversarial Reasoning) are highly efficient. Deployment strategies must account for both, potentially using rate limiting or other mechanisms to detect high-volume, low-success attacks, while also being vigilant for low-volume, high-impact reasoning-based attacks.
• Closed vs. Open-Box Challenges: The distinction between closed-box (API-only) and open-box (full model access) models impacts defense strategies. For closed-box models, transferability of attacks is a concern, and defenses must rely on black-box or gray-box approaches.

Limitations and Future Considerations:

• Weak Defenses Against Advanced Attacks: Current defenses are notably weak against multi-turn and especially decomposition attacks. This is a major research gap, implying that these powerful attack vectors remain largely unmitigated.
• Scaling Laws Debate: The idea that larger, more capable LLMs might inherently be safer (due to "deliberative alignment" or other emergent properties) is an open question. While some evidence suggests this, the "Swiss Cheese Model" remains the industry standard, indicating skepticism about relying solely on scale for safety.
• Jailbreak Tax: The concept of a "jailbreak tax" suggests that even if a model is jailbroken, the quality or usefulness of the harmful information provided might be low. Future defense strategies could aim to increase this "tax," making jailbreaking less rewarding for attackers.

In essence, the move towards increasingly autonomous AI agents necessitates a proactive, multi-faceted approach to security. The current state is an arms race, where new attack methods rapidly emerge, challenging existing defenses. A deeper understanding of agentic systems' unique vulnerabilities and a commitment to robust, layered defenses are critical for harnessing the benefits of AI while mitigating its profound risks.

Key Takeaways

• Jailbreaking is an Evolving and Pervasive Threat: Despite sophisticated safety training (RLHF, preference optimization), all frontier LLMs remain vulnerable to jailbreaking, with research in this area growing exponentially and demonstrating increasing sophistication of attacks.
• Diverse Attack Methodologies Exist: Attacks range from token-level (e.g., GCG, Adaptive Random Search) which often use "gibberish" suffixes, to prompt-level (e.g., PAIR, TAP, Adversarial Reasoning) which employ conversational or reasoning-based strategies, and advanced multi-turn and decomposition attacks that are particularly potent.
• Defenses Require a Multi-Layered "Swiss Cheese" Approach: Effective defense involves combining various strategies—prompt manipulation (e.g., randomized smoothing), model fine-tuning (e.g., Circuit Breaker, Deliberative Alignment), and external guardrails (e.g., Latent Space Guard, Constitutional Classifiers)—as no single defense is foolproof.
• Agentic Systems Significantly Amplify Risks: As LLMs gain more autonomy and interact with the external world (routers, tool callers, iterators, fully autonomous agents), the attack surface expands dramatically, escalating potential harm from generating objectionable information to facilitating digital crimes, data leakage, and even causing physical damage via real-world robots.
• Significant Gaps in Agent Security and Advanced Attack Defenses: Current defenses are notably weak against multi-turn and decomposition attacks. Furthermore, the security implications and appropriate guardrails for fully autonomous agents remain largely unknown and require urgent, dedicated research before widespread deployment.
• Trade-offs Between Safety and Utility are Inherent: Defense mechanisms often involve compromises, such as the risk of "over-refusal" (rejecting benign queries) or the need for white-box access and retraining, which can reduce a model's overall usefulness or introduce new vulnerabilities.

About the Speaker(s)

The tutorial was presented by a team of experts:

• Hamed Hassani: Focused extensively on the technical details and methodologies of various jailbreaking attacks, providing a deep dive into how these vulnerabilities are exploited.
• Amin Karbasi: Provided the foundational context for LLMs, discussed the motivations behind jailbreaking, and elaborated on the "Swiss Cheese Model" of defenses, including his work at Robust Intelligence.
• Alexander Robey: Traced the historical trajectory of jailbreaking and led the speculative discussion on the security implications of deploying AI agents, highlighting the escalating risks and open research questions in this frontier area.