I made AI agents apply for my Security Team. Then I gave the agents access to AWS.

I made AI agents apply for my Security Team. Then I gave the agents access to AWS.

Speakers: Cole Horsman (KKR)

Conference: fwd:cloudsec

YouTube: https://www.youtube.com/watch?v=oJ1Ts8KZw1I

Overview

In a compelling presentation at fwd:cloudsec, Cole Horsman, an AI security specialist at KKR, unveiled an innovative approach to tackling the pervasive challenge of cloud identity and access management (IAM). The talk, provocatively titled "I made AI agents apply for my Security Team. Then I gave the agents access to AWS," details Horsman's journey in developing and deploying an autonomous, multi-agent system designed to manage and secure AWS IAM at scale. This initiative addresses a critical industry gap: the chronic lack of dedicated resources for IAM security amidst an explosion of machine identities, which often outnumber human identities by a staggering 50-to-1.

Horsman's project moves beyond theoretical discussions, demonstrating a practical framework where AI agents not only identify but actively remediate IAM misconfigurations and vulnerabilities. The core idea is to augment human security teams, providing a continuous, automated mechanism for improving cloud identity posture, a task traditionally bogged down by manual reviews, developer reluctance, and the sheer complexity of modern cloud environments. The talk highlights how these agents, once "hired" through a simulated application process, can deliver tangible security improvements, significantly reducing critical risk scores within days.

The relevance of this work extends to any organization grappling with identity sprawl, permission creep, and the challenge of maintaining least privilege in dynamic cloud infrastructures. By showcasing a multi-agent architecture that integrates red teaming, planning, execution, and anomaly detection capabilities, Horsman provides a blueprint for leveraging advanced AI to achieve a more resilient and automated cloud security posture. This talk is not just about automation; it's about shifting the paradigm of cloud IAM from reactive, periodic reviews to proactive, continuous enforcement, ultimately freeing up human experts for more strategic security initiatives.

Background

▶ Watch: Introduction: Cloud identity problem and lack of resources. (0:00)

The genesis of this project lies in a widely recognized, yet frequently unaddressed, problem within cloud security: the cloud identity issue. Organizations understand the criticality of securing IAM, but often lack the dedicated headcount and resources to manage it effectively. Cole Horsman, drawing from his experience in cloud security at Global Atlantic and later in the vendor space, observed a consistent pattern: teams knew what needed fixing, could identify overly permissive roles, and even demonstrate unused permissions via tools like Access Analyzer, but struggled to implement changes at scale. Manual reviews were often biannual, insufficient to keep pace with the dynamic nature of cloud environments and the proliferation of machine identities. The ratio of machine to human identities can be as high as 50:1, making manual oversight virtually impossible and leading to significant identity debt.

Traditional approaches to IAM security, such as static service control policies (SCPs) and customer-managed policies, provide a baseline but often fall short in addressing the nuances of granular permissions and the continuous drift of configurations. The friction involved in requesting developers to remove unused or risky permissions, even with clear evidence, further exacerbates the problem. This leads to a build-up of technical debt in the form of overly permissive roles, unused access keys, and unpatched attack paths, leaving organizations vulnerable to breaches like the Capital One incident.

Horsman's previous role at Sonrai, a cloud security platform, provided him with access to an API layer that offered fine-grained control over AWS permissions, enabling the concept of a permissions firewall. This experience, combined with the persistent challenge of understaffed security teams, inspired the development of an AI agent system. The goal wasn't to invent a new IAM employee from scratch, but to create an intelligent assistant capable of helping teams fix identity issues, automate remediation, and make incremental, measurable progress in improving cloud identity posture, even without additional human headcount. The project aimed to move beyond simply identifying problems to actively solving them, providing a tangible return on security investment.

Key Findings

▶ Watch: The 'hiring process': critiquing and selecting the best agent. (4:00)

The central finding of Cole Horsman's work is the demonstrable efficacy of a multi-agent AI system in significantly improving cloud IAM security posture within a short timeframe. The experiment revealed that such a system can not only identify but also autonomously remediate complex identity issues, delivering quantifiable risk reduction.

Firstly, the project successfully validated the concept of "hiring" and deploying AI agents for specialized security roles. By treating the agent's initial configuration and prompt engineering as a job application process, Horsman demonstrated a novel way to refine and select the most capable AI model for the task. The chosen agent, scoring 92 out of 100 in a simulated evaluation, highlighted the potential for structured AI development.

Secondly, the multi-agent architecture proved highly effective. By compartmentalizing responsibilities into distinct Red Team, IAM Plan, IAM Apply, and UEBA agents, the system could perform comprehensive security functions: identifying attack paths, prioritizing remediation, executing changes, and detecting anomalies. This modularity allowed for specialized capabilities and improved overall system resilience.

Most notably, the experiment yielded concrete, measurable security improvements. Within just five days of operation in a controlled sandbox environment, the agents reduced the account's arbitrary critical risk score from 82 to 47 moderate. This reduction was directly correlated to actual attack paths and involved tangible actions such as locking down access keys that had been unused for over a year and preventing known breach scenarios (like the Capital One attack) without human intervention. The ability to make incremental changes, such as a set threshold of "three changes per day," addressed the practical concerns of enterprise-level adoption, demonstrating that significant improvements are possible without causing undue friction or disrupting existing operations.

Finally, the project underscored the potential for AI to provide tangible, auditable results for security teams. By generating progress reports and quantifiable risk reductions, the system offered a clear demonstration of value, addressing the consultant's perennial need to "prove why they're still paying me." This outcome is crucial for security leaders seeking to justify investments and communicate the impact of their initiatives to stakeholders, including the CISO.

Technical Deep Dive

▶ Watch: Establishing strict IAM identities for specialized agents. (5:30)

Cole Horsman's approach to automating AWS IAM security involves a sophisticated, multi-agent architecture, designed to address the challenges of identity sprawl and the lack of dedicated security resources. The system is built upon several key technical components and distinct AI agent roles.

The initial phase involved a unique "hiring" process for the foundational AI agent. Horsman created a detailed job requisition for a cloud identity expert, specifying requirements like "20 years of cloud identity experience" and listing skills such as AWS CLI, Service Control Policies, Customer Managed Policies, and Access Analyzer. AI models (including DeepSeek, Anthropic (version 4/6), and OpenAI), leveraging tools like Cursor Cloud Code, were then tasked with generating resumes based on this requisition. These resumes, structured with an identity, career summary (equated to a prompt detailing experience), and skills, were then evaluated. An external reviewer, adapted from Caleb Sima's repository for judging session summaries, provided critiques, allowing agents to iteratively refine their "applications." After several rounds of feedback and sharing results, one agent scored 92 out of 100, deemed suitable for "hire."

Once the core agent capability was established, Horsman focused on operationalizing a multi-agent system within a controlled sandbox AWS environment. A critical initial step was establishing a strict identity for the agents, correlated to their specific tasks, rather than using personal SSO credentials. This was achieved using IAM Roles Anywhere, allowing the deployment of these identities locally within an IDE or on a server, facilitating a "crawl, walk, run" development methodology.

The architecture comprises four primary agent types:

1. Red Team Agent:

• Function: This agent's primary role is to proactively identify known attack paths and convert them into protective IAM permissions.
• Methodology: It scans public repositories for documented attack techniques, referencing work by experts like Rami McCarthy from Wiz (e.g., Capital One breach attack paths).
• Output: It translates these attack paths (e.g., those involving S3 or KMS) into specific IAM permissions that need to be protected or restricted within the environment. This was identified as the most challenging agent to "dial in" due to the difficulty of finding relevant, specific attack path repositories without wasting tokens on broad searches.

1. IAM Plan Agent:

• Function: Acts as the strategist, analyzing the environment and formulating remediation plans.
• Methodology: It has read-only access to the AWS environment and can read findings from the Red Team agent and other sources. It utilizes tools like Access Analyzer to identify unused or overly permissive identities (e.g., those not used in 90 days). It then assigns an arbitrary risk score to roles (e.g., 500 roles in an account) based on factors like the level of access and the identity's last-used time.
• Output: On "day two," it generates a detailed plan of actions to be taken, such as disabling access keys or applying protective permissions, and stores this plan (e.g., in an S3 bucket). A key challenge was fine-tuning this agent to "stay on the rails" and only plan for the intended scope of work.

1. IAM Apply Agent:

• Function: The executor, responsible for implementing the changes planned by the IAM Plan Agent.
• Methodology: This agent possesses IAM write access and takes instructions directly from the Plan Agent. It uses the Sonrai API as an orchestration layer. The Sonrai API is crucial here, as it allows for the application of custom controls and functions as a permissions firewall, enabling granular protection of any permission within AWS. For example, it can convert an "S3 list" permission in a data account into a protected action requiring a Teams or Slack approval from a human.
• Operational Control: The system can be configured to make incremental changes, such as a threshold of "three changes per day," to minimize disruption and friction. It operates without interrupting the Terraform state file, applying permissions directly to roles.

1. UEBA Agent (User and Entity Behavior Analytics):

• Function: Provides real-time anomaly detection and rapid response.
• Methodology: Integrates with security findings from platforms like Wiz and Security Hub. It monitors for anomalies or "sus" activities based on indicators of compromise (IOCs).
• Response: Implements a tiered response: for critical anomalies, it "asks forgiveness, not permission" and immediately quarantines the identity; for high-severity issues, it "asks permission" before quarantining.

The underlying infrastructure also leverages Cedar policy for advanced allow/deny statements and redacting unnecessary access, building a multi-threaded, defense-in-depth foundation. This comprehensive architecture allows for continuous monitoring, proactive threat mitigation, and automated remediation, addressing the accumulated "identity debt" that parallels unpatched CVEs.

Demo / Proof of Concept

▶ Watch: The IAM Plan agent: converting research, risk scoring. (7:00)

While the talk did not feature a live, real-time demonstration of the agents in action, the entire presentation served as a detailed proof of concept by outlining the "experiment" and presenting its concrete, quantifiable results within a controlled sandbox environment. Cole Horsman meticulously described the setup, the operational flow of the multi-agent system, and the significant security improvements achieved over a short period.

The core demonstration of the system's capability was encapsulated in the five-day results provided by Horsman. Within this timeframe, the AI agents successfully:

• Reduced Critical Risk Score: The account's arbitrary critical risk score, which was correlated to actual attack paths, was dramatically reduced from 82 to 47 moderate. This reduction was a direct measure of the agents' effectiveness in identifying and remediating high-impact vulnerabilities.
• Locked Down Unused Access Keys: The agents identified and locked down access keys that had not been used in over a year, eliminating a common source of security risk and demonstrating their ability to address "messy things" that often accumulate in cloud environments.
• Mitigated Known Attack Paths: Crucially, the system proved capable of preventing specific, well-known attack paths, such as the Capital One breach scenario, without human intervention. Horsman emphasized that in the tested account, "no identity is able to bypass this without a human in the loop approval," showcasing the system's ability to enforce robust security gates for critical actions.

This "demo" underscored the feasibility of an automated approach to IAM security, providing tangible evidence of risk reduction and improved posture. It highlighted that the agents could make incremental, non-disruptive changes (e.g., setting a threshold of "three changes per day") while still achieving substantial security gains. The results were presented as a measurable outcome that could be reported to a CISO, demonstrating the value and effectiveness of the AI-driven security team. The speaker explicitly stated, "Hey, week one I accomplished these things. I reduced your risk score by X%. And it gives something tangible like a result that you could you could also approve." This narrative served as a powerful validation of the system's potential.

Defensive Implications

▶ Watch: The IAM Apply agent: implementing planned security changes. (8:00)

The deployment of AI agents for cloud IAM security, as demonstrated by Cole Horsman, carries profound defensive implications for organizations struggling with identity sprawl, permission creep, and resource constraints.

1. Automated IAM Remediation at Scale: The most significant implication is the ability to automate the continuous remediation of IAM misconfigurations. Instead of biannual, manual reviews, security teams can leverage AI agents to continuously monitor, identify, and fix issues. This directly addresses the "identity debt" that accrues over time, much like unpatched CVEs, by making incremental but consistent improvements (e.g., 3 changes per day) without human overhead.

1. Proactive Threat Mitigation: The Red Team Agent paradigm allows organizations to proactively translate known attack paths (e.g., from past breaches like Capital One) into preventative IAM controls. This shifts the defensive posture from reactive to predictive, ensuring that specific, high-impact attack vectors are blocked before they can be exploited.

1. Enhanced Visibility and Risk Scoring: The IAM Plan Agent provides continuous risk scoring of identities based on access levels and usage patterns (e.g., unused for 90 days), offering a dynamic and data-driven view of the organization's identity posture. This enables security teams to prioritize remediation efforts based on actual risk rather than anecdotal evidence.

1. Real-time Anomaly Detection and Response: The UEBA Agent integrates with existing security tooling (Wiz, Security Hub) to detect anomalous behavior and potential indicators of compromise (IOCs) in real-time. Its ability to automatically quarantine identities based on criticality (asking forgiveness vs. permission) drastically reduces response times for emerging threats.

1. Defense-in-Depth Reinforcement: The multi-agent system complements existing security controls like Service Control Policies (SCPs) and Customer Managed Policies. By using a permissions firewall (via Sonrai API) and Cedar policies, it adds granular, dynamic enforcement layers that go beyond static policies, creating a more robust, multi-threaded defense architecture.

1. Addressing Headcount Shortages: For organizations lacking dedicated IAM security headcount, these agents act as a force multiplier. They enable smaller teams to manage complex cloud identity environments effectively, freeing up human experts to focus on strategic initiatives, architectural design, and complex incident response rather than repetitive, time-consuming remediation tasks.

1. Quantifiable Security Improvements: The ability to report tangible metrics, such as the reduction of critical risk scores (e.g., from 82 to 47 moderate in 5 days), provides CISOs with clear evidence of security posture improvement. This facilitates better communication with stakeholders and justifies security investments.

1. Human-in-the-Loop Control: Importantly, the system is designed with a human-in-the-loop approval mechanism (e.g., Slack/Teams approvals for sensitive actions). This ensures that critical decisions are still reviewed by human experts, balancing automation with necessary oversight and trust. It acknowledges that while agents can deploy controls, the ultimate responsibility and understanding of permissions depth remain with human security professionals.

Key Takeaways

• AI Agents Bridge the IAM Headcount Gap: AI-driven multi-agent systems can effectively automate complex cloud IAM security tasks, providing a scalable solution for organizations lacking dedicated identity security resources and addressing the challenge of managing a high ratio of machine identities to human staff.
• Multi-Agent Architecture for Comprehensive Security: A modular approach, utilizing specialized agents for red teaming, planning, execution, and anomaly detection (Red Team, IAM Plan, IAM Apply, UEBA Agents), allows for a holistic and robust defense-in-depth strategy against cloud identity threats.
• Quantifiable Risk Reduction is Achievable: The experiment demonstrated significant, measurable improvements in cloud security posture, reducing critical risk scores (e.g., from 82 to 47 moderate in 5 days) and mitigating known attack paths, providing clear ROI for AI-driven security initiatives.
• Automation Augments, Not Replaces, Human Expertise: While AI agents can automate remediation and continuous monitoring, human oversight remains crucial for approving critical changes, understanding the depth of permissions, and making strategic security decisions. The agents act as powerful assistants, not autonomous replacements.
• Leverage Existing Tools with AI Orchestration: The system effectively integrates existing AWS security tools like Access Analyzer, Service Control Policies, and external security findings (Wiz, Security Hub) with an orchestration layer (Sonrai API) and advanced policy engines (Cedar), enhancing their capabilities through AI.
• Address "Identity Debt" Proactively: Just as organizations manage CVEs, the accumulation of overly permissive roles and unused access is a critical "identity debt." AI agents provide a mechanism for continuous, incremental remediation, preventing this debt from becoming a major security liability.

About the Speaker(s)

Cole Horsman is a distinguished professional in the field of cloud and AI security, currently focusing on AI security at KKR. His career path reflects a deep understanding of cloud environments and the evolving challenges within them. Prior to his current role, Horsman gained significant experience in cloud security at Global Atlantic for several years. He then ventured into the vendor space, a detour he humorously admits wasn't his forte in sales, before returning to KKR. His work at Sonrai, a cloud security platform, was particularly influential, providing him with access to API layers that enabled the fine-grained control over AWS permissions, which became a foundational element of the multi-agent system discussed in his talk. Horsman's expertise lies in identifying pervasive security problems, such as identity sprawl and the lack of dedicated resources for IAM, and developing innovative, automated solutions leveraging artificial intelligence.