Shaping the Future of AI Security Through Collaboration

Shaping the Future of AI Security Through Collaboration

Speakers: Jamil Jaffer (Paladin Capital Group), Jason Clinton (Anthropic), Matt Knight (OpenAI), Sandra Joyce (Google)

Event: RSA Conference 2025 — April 28–May 1, 2025, Moscone Center, San Francisco

Watch: YouTube — Shaping the Future of AI Security Through Collaboration

Reading time: ~8 minutes

TL;DR

Leaders from Anthropic, OpenAI, and Google Threat Intelligence joined a policy panel at RSA Conference 2025 to assess how AI is reshaping the cybersecurity balance of power between attackers and defenders. The consensus: defenders currently hold the advantage, but that window is closing fast and must be fortified through rigorous measurement, cross-sector collaboration, and sustained investment in AI-native security tooling. Inaction now could cost the security community its first-mover edge.

Introduction

At 8:30 on a Tuesday morning, the Policy Stage at RSA Conference 2025 was packed. The draw was a conversation that sits at the center of nearly every security team's planning horizon: how should the industry and government work together to navigate a world where artificial intelligence is simultaneously the most powerful tool in the defender's arsenal and a growing asset for adversaries?

Moderated by Jamil Jaffer, venture partner at Paladin Capital Group and founder of the National Security Institute at George Mason University, the panel brought together three CISOs and threat intelligence leaders whose organizations collectively shape the trajectory of AI development globally. The session, titled "Shaping the Future of AI Security Through Collaboration," covered attacker vs. defender AI symmetry, real-world threat observations, regulatory posture, and the long-term structural changes AI is poised to bring to security operations.

The overarching message was cautiously optimistic but urgent: defenders are ahead today, and the industry must act decisively to stay that way.

AI Capabilities Are Doubling Every Ten Months — and Defenders Must Outpace That Curve

OpenAI's Matt Knight opened the technical discussion by recounting the transformation he has witnessed since joining the company in summer 2020, shortly before the release of GPT-3. Where GPT-3 was "a curiosity," current frontier models represent a sea change in what security teams can accomplish — and what adversaries might eventually attempt.

▶ Watch: Defender vs. attacker AI symmetry (02:30)

Knight pointed to OpenAI's preparedness framework, released in early 2023, as a foundational piece of infrastructure for evaluating model risk across multiple threat categories. That framework replaces subjective assessments — what Knight calls "vibes" — with actual empirical measurement. And the data from those evaluations is striking: cybersecurity capabilities in frontier models are roughly doubling every ten months.

That rate of improvement cuts both ways. Applied defensively, it means AI can increasingly automate incident summarization, reduce SOC toil, flag anomalous behavior at scale, and eventually run fully autonomous penetration tests against an organization's own infrastructure before attackers do. Knight's team at OpenAI has been using language models internally to streamline everything from incident post-incident reviews to organizational communication during live responses.

But the doubling curve also means the window during which defenders retain a structural advantage is finite. "It's very important that we as a community continue to test and monitor these risks so that we can ensure that the balance of power continues to support the defenders," Knight said.

▶ Watch: AI capabilities doubling every ten months (06:00)

Threat Actors Are Already Using AI — Just Not in the Ways Most Fear

Sandra Joyce, VP of Google Threat Intelligence, offered a grounded assessment of what adversaries are actually doing with AI today, drawing on her team's monitoring of thousands of personas across the underground and dark web. The honest answer: threat actors are using AI for deep fakes, for jailbroken chatbot offerings, and for accelerating social engineering — but no actor has yet demonstrated a genuinely novel capability that exceeds what a skilled human operative could accomplish.

▶ Watch: What threat actors are doing with AI today (06:30)

"When I ask my team to look at what threat actors are actually doing today, not lurching from this headline to this headline or this scary scenario," Joyce explained, "what my team came back with is they're using it for deep fakes in the underground. They're using it to sell so-called jailbreak versions of some of the chatbots."

The key caveat, however, is that defenders still control the technology roadmaps and the guardrails — and that control is what gives the good side its current lead. Joyce was explicit that it is "just a matter of time" before adversaries gain access to the same unconstrained capabilities that internal Google security teams use for tasks like malware reversal, vulnerability scanning, and enhanced fuzzing. The blog post she referenced — Google's "Big Sleep" research, in which an LLM was used to discover an actual software vulnerability — illustrates precisely why the defender advantage must be exploited aggressively now.

▶ Watch: Google's Big Sleep vulnerability discovery (08:00)

Democratization of Elite Coding Talent Is the Threat to Watch

Jason Clinton, CISO at Anthropic, zeroed in on what he described as perhaps the most consequential structural shift AI introduces to the threat landscape: the democratization of top-tier software engineering capability.

▶ Watch: AI democratizing access to elite coding talent (10:00)

"What is happening in AI right now is that we are democratizing access to the very top-tier computer science talent in the world," Clinton said. "That means that your tier three actors who have typically been your script kiddies now have access to coding agents that can write code at the level of someone who's maybe a very competent programmer in a Silicon Valley context."

Anthropic had itself observed ransomware being developed on its platform — a threat that was identified and shut down — but Clinton's larger concern was forward-looking. As agentic AI models become capable of running fully automated penetration tests, conducting application security reviews, and preventing bugs from shipping in the first place, the nature of the security profession itself will shift. That future, Clinton suggested, is closer than most security teams are currently planning for.

Clinton also noted that Anthropic had published transparency reporting — released the week before RSA — detailing what influence operations, business email compromise augmentation, and AI-as-a-service offerings the company's models had actually been used for. That kind of empirical, public disclosure, he argued, is essential for the broader community to form accurate threat models rather than relying on speculation.

The Defender-Side Playbook: Measurement, Transparency, and Shared Infrastructure

Across all three panelists, a consistent theme emerged: the security community's response to AI-driven threats must be grounded in measurement, not rhetoric. Knight's ten-months-per-doubling figure, Joyce's underground persona monitoring, and Clinton's published blog data all reflect a shared conviction that vague threat narratives do not produce actionable defense.

▶ Watch: The case for rigorous AI risk measurement (04:00)

The session also probed what government-industry collaboration should look like in practice. Jaffer framed the challenge in terms of existing structural gaps: the security community has world-class threat intelligence but limited mechanisms for translating it into coordinated defensive action. Clinton pointed to Anthropic's responsible scaling policy as one model for institutionalizing the feedback loop between capability development and security investment, requiring that each new capability tier be matched with demonstrated safety measures before deployment.

Joyce argued that the companies building AI — and those defending against its misuse — hold a temporary but critical advantage simply by virtue of controlling the technology and its constraints. Maintaining that advantage demands that organizations on the defender side invest in AI-native tooling now, while the capability gap still favors them, rather than treating AI as an incremental add-on to existing security stacks.

The panel agreed that the current moment is best understood as a "pre-phase" — a window in which the structural conditions favor defenders but that window is not permanent. The next phase, when adversaries gain access to equivalent or unconstrained capabilities, will require a defensive infrastructure that is already mature.

Notable Quotes

"If we look at our preparedness evals for cybersecurity testing, we see that roughly every ten months, the cybersecurity capabilities in models double." — Matt Knight, OpenAI

"Right now, I think we're in this pre-phase where we have a lot of time to do some innovation on the defender side. And I would say that right now, that advantage is really truly rooted on the defender side." — Sandra Joyce, Google

"What is happening in AI right now is that we are democratizing access to the very top-tier computer science talent in the world — and that means that your tier three actors who have typically been your script kiddies now have access to coding agents that can write code at the level of someone who's maybe a very competent programmer in a Silicon Valley context." — Jason Clinton, Anthropic

"We need to make our investments so that we keep ahead of the attackers." — Sandra Joyce, Google

Key Takeaways

1. AI capabilities in models are doubling roughly every ten months — a rate that applies equally to offensive and defensive applications, making sustained investment in measurement and evaluation frameworks non-negotiable.

1. Defenders currently hold the advantage because they control the technology roadmaps and guardrails, but that lead is finite and eroding; the time to build AI-native defensive infrastructure is now.

1. Threat actors are not yet doing anything with AI that they couldn't do more slowly without it — the primary observed uses remain social engineering augmentation, influence operations, and deep fakes, not novel technical exploitation.

1. The democratization of elite coding capability is the most structurally significant long-term shift: AI is collapsing the skill barrier that previously separated script kiddies from sophisticated threat actors.

1. Transparency and measurement are foundational — organizations like Anthropic and OpenAI publishing empirical threat data, and Google's Big Sleep vulnerability research, demonstrate the kind of collaborative, evidence-based posture the industry needs to sustain the defender advantage.

1. Public-private collaboration requires active investment, not just coordination agreements; the structural gap between threat intelligence and coordinated defensive action must be closed before the capability window closes.