Agentify OSINT: Transforming Security News Feeds Into Actionable Intelligence

Agentify OSINT: Transforming Security News Feeds Into Actionable Intelligence

Speakers: Cristian Miron (Security Engineer, UiPath); Valentin Vatui (Cyber Security Engineer, UiPath)

Conference: DefCamp

YouTube: https://www.youtube.com/watch?v=wYK_KVQDpl0

Overview

In an era of escalating cyber threats and resource constraints, security teams often find themselves in a reactive posture, struggling to keep pace with the sheer volume of emerging vulnerabilities and attack campaigns. The DefCamp talk "Agentify OSINT: Transforming Security News Feeds Into Actionable Intelligence" by Cristian Miron and Valentin Vatui, both Cyber Security Engineers at UiPath, addresses this critical challenge head-on. Their presentation unveils an innovative, automation-driven approach that leverages Large Language Model (LLM)-based agents to proactively transform raw security news and Open Source Intelligence (OSINT) into immediate, actionable threat intelligence.

The core of the talk revolves around the development and implementation of a sophisticated workflow designed to overcome the limitations of small security teams, specifically the inability to scale alongside an increasing influx of external security notifications. Miron and Vatui demonstrate how their "Agentify" system automates the laborious process of analyzing security articles, extracting Indicators of Compromise (IOCs) and Tactics, Techniques, and Procedures (TTPs), querying internal log sources, and generating comprehensive threat hunting reports in mere minutes. This not only drastically reduces manual effort but also enables security teams to shift from a perpetually reactive stance to a more proactive and predictable threat hunting operation.

This talk is particularly significant for its practical application of cutting-edge AI technologies, specifically LLMs, in a real-world security context. It showcases how intelligent automation can democratize advanced threat intelligence capabilities, allowing organizations to efficiently determine their exposure to newly reported attacks, prioritize remediation efforts, and continuously enhance their overall security posture without the need for additional headcount. By detailing the architecture, agent functionalities, and tangible benefits, Miron and Vatui provide a compelling blueprint for how security teams can leverage automation to build resilience in the face of an ever-evolving threat landscape.

Background

▶ Watch: Manual threat hunting workflow before automation (5:00)

The impetus for the Agentify OSINT project stemmed from a common predicament faced by many small security teams: the struggle to balance day-to-day operational tasks like security triage and incident response with the critical but often deprioritized activities of proactive threat hunting. As Miron explained, threat hunting is defined as the proactive search for hidden threats within one's environment, encompassing data analytics, behavioral analytics, and threat intelligence. However, the operational realities often made this ideal difficult to achieve.

A significant contributing factor to this imbalance was the increasing volume of external notifications from users or management, often phrased as, "Hey, I was reading this news article talking about this or that attack. Are we impacted by this?" While such inquiries are a positive sign of industry engagement, they create a considerable amount of "noise," leading to constant context switching for security analysts. This often results in the deprioritization of higher-impact tasks and a feeling that "everything is urgent." The most straightforward solution—adding headcount—was not feasible due to prevailing economic constraints in the tech industry. Instead of adding people, UiPath's team opted to "add agents."

This decision, however, introduced its own set of challenges. The automated agents needed to offer broad coverage, checking every log source and data point, similar to what a human engineer would do. Simultaneously, they had to be highly specific in their queries to ensure relevance in the final reports. Language was another critical consideration; the agents needed to communicate clearly and precisely to avoid creating a false sense of security or generating misleading reports. Efficiency was paramount: agents should not run checks for IOC types not present in an article, and the system needed a mechanism to prevent re-analysis of articles that had already been processed. The manual process before Agentify was time-consuming and inefficient: a user would ask about an attack, an analyst would read the article, manually note down IOCs, consult internal documentation for relevant log tables and fields, craft KQL (Kusto Query Language) queries, run them in their ADX instance, and then potentially create detection rules. This entire process could take at least an hour, often for attacks that ultimately had zero impact on the infrastructure, diverting valuable time from critical backlog projects. The need for a scalable, efficient, and intelligent automation solution was clear.

Key Findings

▶ Watch: Demonstration: Building and using an LLM agent (10:00)

The implementation of the Agentify OSINT system yielded several transformative key findings and accomplishments for UiPath's security team, fundamentally altering their approach to threat hunting and security operations.

Firstly, the system delivered a reusable workflow capable of validating an organization's exposure to specific CVEs or attacks. This workflow's crowning achievement is its ability to generate a comprehensive threat hunting report in just 5 minutes from the start of execution. This dramatic reduction in response time is a game-changer, allowing for near real-time assessment of emerging threats.

Secondly, by offloading the initial triage and analysis of numerous security articles to these automated agents, the threat hunting team achieved a more predictable effort. This newfound predictability frees up human analysts from repetitive, low-value tasks, enabling them to focus on continuous professional development and specialize in activities that generate higher value for the company. The result is a direct increase in the overall security posture, as skilled professionals can dedicate their expertise to strategic, complex challenges rather than operational noise.

A significant psychological and operational benefit was the elimination of the "everything is urgent" assumption. Instead of being constantly interrupted by urgent requests to check new articles, the team now receives targeted alerts only when an agent identifies a genuine, high-impact threat requiring human attention. This allows analysts to maintain focus on their planned work, reducing context switching and improving productivity.

Furthermore, the system's integrated database provides crucial capabilities for tracking previous executions and generating metrics for management. This not only prevents the redundant analysis of already processed articles but also addresses the challenge of evolving threat intelligence. If an article is updated with new information or IOCs, the database allows the team to verify if their previous analysis was based on the latest data, ensuring they are always working with current and relevant intelligence. This unexpected benefit ensures that the team's threat hunting efforts remain agile and informed.

Finally, the talk highlighted a significant gain in efficiency and focus. While exact metrics for hours saved were not precisely quantified, the speakers estimated at least 10 hours per week of saved time. More importantly, the system drastically reduced the cognitive load and context switching that previously plagued the team, allowing them to focus uninterruptedly on core projects and investigations.

Technical Deep Dive

▶ Watch: Workflow execution: Agents running in parallel (16:50)

The Agentify OSINT workflow is built upon a sophisticated architecture that integrates several key technologies to achieve its automated threat hunting capabilities. At its heart are LLM-based agents, which possess both a reasoning capacity and the ability to call other tools during their execution.

The overall system architecture comprises:

• Enterprise Messaging Platform: This serves as the primary user interface. Users interact with an app within this platform, submitting a link to an article for analysis. This strategic placement ensures accessibility and ease of use, as most users spend the majority of their time here.
• LLM-based Agents: These are the core intelligent components, designed to perform specific tasks within the threat hunting process.
• Query Tools: Essential for validating IOCs against an organization's internal logs and telemetry. In UiPath's implementation, this involves running KQL queries against their ADX instance.
• Robotic Process Automation (RPA): Used for specific actions requiring consistent and repeatable input/output, ensuring reliability in data handling and integration with other systems.
• Database: A crucial component for tracking previous executions, providing performance metrics, and preventing duplicate analysis of articles. It also helps manage updates to articles, ensuring the latest data is always used.
• Agent Orchestration Platform (Maestro): This platform ties all the disparate components and agents together, orchestrating their parallel and sequential execution to form a cohesive workflow.

The speakers provided an illustrative example of how an agent is built and instructed. An agent starts with a system prompt that defines its role and desired output characteristics (e.g., "its role is within the SOC department" and "answers to be as clear as possible"). A user prompt provides the specific query (e.g., "tell us how many authentication events happened in our infrastructure in the last two hours"). The agent is given access to specific tools, such as a tool to execute KQL queries and another for internal documentation of the logging system. When executed, the agent first consults the internal documentation tool to identify the relevant table (e.g., signinlogs) for authentication events. Subsequently, it utilizes the KQL query tool to execute a query in the log system, quickly revealing findings such as "more than 70k authentication events in the last two hours."

The Agentify workflow incorporates several specialized agents, each with a distinct role:

1. Article Parser Agent: This initial agent reads the provided security article and is responsible for identifying and extracting various Indicators of Compromise (IOCs). These include public IP addresses, domains, email addresses, file hashes, CVE IDs, and SBOM (Software Bill of Materials) packages.
2. Enricher Agents (Six Types): Following the Article Parser, there are six specialized Enricher Agents, one for each type of IOC identified. These agents receive the extracted IOCs and generate specific KQL queries designed to find potential hits for those indicators within the organization's ADX instance.
3. Web Searcher Agent: This agent reads the article, extracts IOCs, TTPs, and other relevant details like attack techniques and actor names. It then searches the broader internet for additional articles and intelligence discussing the same topic, aiming to provide a more comprehensive threat overview in the final report.
4. Hunting Engineer Agent: This agent focuses on TTPs. It reads the article, extracts and validates the TTPs mentioned, and then translates these into KQL queries to run against the ADX instance. Its goal is to determine if the investigated threat's TTPs have any observable presence or activity within the organization's environment.
5. Level Two Security Analyst Agent: This is described as the central hub of the workflow. It receives and synthesizes the outputs from all preceding agents. Based on this aggregated information, it performs the core investigation, determines if the organization is impacted or if the threat is active, and provides security recommendations, including suggestions for new detection rules.
6. Documentation Agent: Taking the investigation results from the Level Two Security Analyst, this agent is responsible for compiling a detailed threat hunting report. This report includes an executive summary, a technical analysis of found TTPs, a list of IOCs, an impact assessment, the KQL queries used, and mitigation/remediation recommendations.

The workflow executes in a highly parallel fashion. The Article Parser, Web Searcher, and Hunting Engineer agents initiate their tasks concurrently. Once the Article Parser finishes, the Enricher agents start their parallel execution. Importantly, the system is efficient; if a specific IOC type (e.g., file hash) is not present in the article, the corresponding Enricher Agent's execution is skipped. All agent outputs converge at the Level Two Security Analyst for investigation, which then passes its conclusions to the Documentation Agent. Finally, a small RPA component integrates with the organization's documentation tool (e.g., Confluence) to create a nicely formatted page for the report, making it easily accessible to the team.

Demo / Proof of Concept

▶ Watch: Example of the automated threat hunting report (18:50)

The core demonstration of the Agentify OSINT system centered around its ability to transform a raw security news article into a fully detailed and actionable threat hunting report within a remarkably short timeframe. The speakers showcased the end result of the intricate workflow they had just described, emphasizing the practical output that makes this automation so valuable.

Starting with any article read online, the system generates a comprehensive threat hunting report in approximately five minutes. This report is not a superficial summary but a deep dive into the attack's specifics and its potential impact on the organization. The contents of the generated report are extensive and include:

• Executive Summary: A high-level overview of the threat and the findings.
• Threat Overview: Detailed context about the attack.
• Threat Actor Attribution: If available in the source article or discovered by the Web Searcher agent.
• MITRE Mapping: A mapping of the identified TTPs to the MITRE ATT&CK framework, providing standardized context for the attack techniques.
• Timeline: A chronological representation of events related to the threat, if applicable.
• List of IOCs Identified: A precise enumeration of all extracted indicators, such as malicious IPs, domains, file hashes, and CVEs. Crucially, the report also specifies which IOCs led to hits within the organization's environment.
• Impact Assessment: A clear statement on the organization's exposure, detailing, for example, that "two devices in our organization might have been impacted" and based on which specific IOCs. This immediate insight allows for swift targeted remediation.
• Detection Query: A pre-built KQL query that can be directly implemented into the organization's SIEM or log analysis platform to detect future occurrences of the identified threat.
• Next Action Items: A bulleted list of recommended follow-up steps for the security team to further investigate, mitigate, and protect against the threat.

The speakers highlighted that the level of detail and comprehensive analysis in these automated reports often surpasses what a human analyst could achieve manually, especially under time pressure. As Cristian Miron humorously noted, the report serves as a "constant reminder of my failures" because he would "never ever go into this many details when it comes to building a threat hunting report" due to time constraints and the sheer effort involved. The demonstration effectively underscored the system's capability to deliver high-quality, actionable intelligence rapidly and consistently, drastically improving the security team's ability to respond to emerging threats.

Defensive Implications

▶ Watch: Q&A: Time savings and avoiding context switching (25:00)

The Agentify OSINT system presents several profound implications for security defenders, offering a blueprint for enhancing operational efficiency, proactive defense, and overall security posture in a resource-constrained environment.

Firstly, the most significant implication is the imperative to embrace intelligent automation. In an era where headcount cannot keep pace with the volume and velocity of cyber threats, LLM-based agents provide a scalable solution. Defenders should explore how similar agentic workflows can automate repetitive, time-consuming tasks, freeing up human analysts for higher-order problem-solving and strategic initiatives.

Secondly, the talk strongly advocates for specializing agents. Rather than monolithic AI systems, defenders should design agents with specific roles and narrow focuses. This approach not only boosts efficiency by avoiding unnecessary computations (e.g., skipping checks for IOC types not present in an article) but also improves the relevance and accuracy of their output by allowing for highly specific prompts and instructions.

Thirdly, providing agents with extensive tools and context is crucial. The "smarter" an agent becomes, the more data sources (log sources, internal documentation, threat intelligence feeds) and tools (KQL executors, web searchers, MITRE mappers) it has access to. Defenders should prioritize integrating their security tools and data lakes with these automated systems to maximize the agents' analytical capabilities and broaden their coverage.

The shift from reactive to proactive threat hunting is a direct benefit. By automating the initial triage and analysis of OSINT, security teams can dedicate more time to deep, continuous threat hunting, developing specialized skills, and focusing on high-value projects that genuinely improve the organization's security posture. This reduces the constant "firefighting" and context switching that often plagues security operations.

Defenders should also consider implementing feedback loops for continuous improvement. As discussed in the Q&A, a mechanism for human review and correction of agent outputs—where an analyst can flag inaccuracies or suggest better phrasing—is vital for refining agent performance and ensuring trust in the system. This iterative process, possibly incorporating "self-improving" agents that adapt based on query failures, is key to long-term success.

Future enhancements discussed, such as automating the publishing of detections to Git, highlight another critical defensive implication: seamless integration of threat intelligence into the detection pipeline. Automatically pushing validated detection queries to a SIEM via Git ensures that newly identified threats are monitored almost immediately, drastically shortening the detection gap.

Finally, the challenge of resource usage on SIEMs/data lakes when agents run numerous parallel queries is a practical consideration. Defenders must ensure that automated queries are well-structured, time-bound, and optimized to prevent overwhelming existing infrastructure. While the speakers noted minimal noticeable impact with their optimized queries, this remains a design consideration for any similar implementation. Furthermore, establishing a system for IOC confidence scoring and handling contradictory OSINT sources would be valuable additions, ensuring that automated findings are robust and reliable.

Key Takeaways

• Specialize Agents for Efficiency and Relevance: Design LLM-based agents with narrow, specific roles to optimize their performance, avoid unnecessary computations, and ensure their outputs are highly relevant to the task at hand.
• Maximize Tools and Context for Agent Intelligence: Provide agents with access to as many internal and external data sources (log systems, documentation, web search) and query tools as possible to enhance their analytical capabilities and confidence in their reports.
• Transform Reactive Security into Proactive Threat Hunting: Leverage automation to offload initial OSINT analysis and triage, enabling human security teams to focus on higher-value, proactive threat hunting, continuous development, and strategic security improvements.
• Generate Comprehensive Threat Hunting Reports in Minutes: Implement agentic workflows to produce detailed, actionable threat hunting reports, complete with IOCs, TTPs, impact assessments, and remediation recommendations, within five minutes, drastically reducing response times.
• Reduce Context Switching and Achieve Predictable Workflows: Automation minimizes interruptions from "urgent" security news, allowing security professionals to maintain focus, reduce cognitive load, and achieve a more predictable and productive workday.
• Embrace Continuous Improvement and Feedback Loops: Integrate mechanisms for human review and automated self-correction to continuously refine agent accuracy, address issues like hallucinations or query failures, and ensure the long-term reliability of the automated system.

About the Speaker(s)

Cristian Miron is a Security Engineer at UiPath. With 14 years of experience in IT, 11 of which have been dedicated to security, Cristian has a deep understanding of the industry. Early in his career, he was influenced by a mentor who demonstrated the profound benefits of automation in security, leading him to focus his career path in this direction. His expertise lies in leveraging automation to solve complex security challenges.

Valentin Vatui is a Cyber Security Engineer at UiPath. He brings over 8 years of experience in the cybersecurity domain, with a specialized focus on critical areas such as threat hunting, security investigations, security operations, and incident response. Valentin also possesses strong capabilities in security engineering, contributing to the development and implementation of robust security solutions.