Why Vulnerability Assessment Isn't Sufficient for Attack Surface Management

Why Vulnerability Assessment Isn't Sufficient for Attack Surface Management

Speakers: Catalin Neacsu; Darius Iakabos (Lead Solutions Architect, Arcana AI)

Conference: DefCamp

YouTube: https://www.youtube.com/watch?v=_kuOczQVCj4

Overview

In an era where cyber threats constantly evolve, organizations frequently find themselves operating in a false sense of security, relying solely on traditional vulnerability assessment for their attack surface management. This talk, presented by Catalin Neacsu and Darius Iakabos at DefCamp, challenges this conventional wisdom, arguing that an over-reliance on scan-only systems leaves critical blind spots. The speakers introduce a sophisticated approach that integrates external threat intelligence and agentic AI workflows with advanced machine learning decision models to provide a more comprehensive and proactive defense strategy.

The core premise is that an organization's true attack surface extends far beyond its network perimeter, encompassing exposures like stolen credentials, dark web mentions, and misconfigurations that traditional scanners often miss. Catalin Neacsu, with his extensive background in security operations and threat intelligence, teams up with Darius Iakabos, a lead solutions architect at Arcana AI specializing in decision models and agentic workflows, to demonstrate how their platforms—No Intel's Griffin and Arcana AI—can bridge these visibility gaps and automate intelligent responses.

This presentation is crucial for security professionals grappling with the limitations of existing vulnerability management tools and the overwhelming volume of security alerts. By showcasing how agentic AI can not only detect a broader range of threats but also make informed, automated decisions, Neacsu and Iakabos offer a compelling vision for the future of security operations, moving from reactive scanning to proactive, context-aware defense.

Background

▶ Watch: Vulnerability scanners alone create a false comfort zone (1:30)

The pervasive challenge in cybersecurity today is the ever-expanding and dynamic nature of the attack surface. Many organizations operate under the misconception that regular vulnerability scanning provides adequate coverage. However, as the speakers emphasize, "no CVs doesn't necessarily mean no exposures." Traditional scanners are inherently limited; they often lack real-time threat intelligence concerning ongoing campaigns, active threat actors, and the availability of public exploit code. This delay in signature deployment means attackers can often develop and deploy exploits for newly disclosed vulnerabilities faster than security teams can even detect them.

Furthermore, attackers are not confined to the network perimeter. They actively map organizations using internet noise, acquire leaked data from dark web marketplaces and Telegram channels, and exploit any opportunity to gain an initial foothold. Statistics from the 2024 Verizon report highlight this reality, with stolen credentials remaining the top attacker entry point, accounting for 24% of initial entry vectors. This underscores a critical imbalance: defenders must be perfect every time, while attackers only need one vulnerability to succeed. This necessitates a shift in mindset, urging defenders to adopt an adversary's perspective.

The consequence of these blind spots is significant business risk. Unseen threats equate to unsafe environments. The analogy drawn is that "not knowing the law does not absolve you from not abiding to it"; similarly, ignorance of a threat does not protect an organization from its impact. The sheer volume of security alerts also overwhelms defenders, making it difficult to prioritize and act effectively. This context sets the stage for the introduction of advanced solutions that can provide comprehensive visibility and intelligent decision-making capabilities, moving beyond the reactive and often insufficient paradigm of scan-only vulnerability assessment.

Key Findings

▶ Watch: Arcana AI: experts train neural networks for security decisions (8:20)

The central finding of this talk is that traditional vulnerability assessment is fundamentally insufficient for comprehensive attack surface management. The speakers highlight that relying solely on scanners creates a "false comfort zone," as these tools often miss critical exposures stemming from stolen credentials, dark web data, and real-time threat campaigns. The core problem is a lack of context, real-time intelligence, and the inability to quickly adapt to rapidly evolving threat landscapes.

To address this, the presentation introduces a two-pronged solution leveraging agentic AI and machine learning:

1. Unified Visibility and Contextual Intelligence (No Intel's Griffin): The Griffin SaaS platform aims to consolidate visibility from diverse security domains, including traditional vulnerability scanning, digital exposure monitoring, deep dark web threat intelligence, and GitHub repository analysis. This unified approach transforms disparate data into actionable insights. A key innovation is a built-in chatbot that allows security defenders to converse with their own organizational data, powered by Large Language Models (LLMs), Retrieval Augmented Generation (RAG), and Multi-tool Coordination Protocol (MCP) technologies. This provides enhanced context and data retrieval capabilities.

1. Automated, Deterministic Decision-Making (Arcana AI): Arcana AI introduces a platform for subject matter experts to train their own deep neural networks as decision models. Unlike LLMs, which can be non-deterministic and prone to "hallucination," Arcana's models are trained on specific company data, ensuring accuracy and trust for critical security decisions. These models can triage alerts from detection platforms, make predictions (e.g., investigate, archive, specific remediation), and trigger agentic workflows to automate responses. This separation of LLMs for data gathering/reporting and specialized neural networks for decision-making is a critical distinction presented.

By integrating these capabilities, the speakers propose a paradigm shift from manual detection-and-report cycles to an automated, intelligent, and context-aware security operation that can detect, decide, and respond with unprecedented speed and accuracy.

Technical Deep Dive

▶ Watch: Live demo: Griffin chatbot answers security questions using RAG and MCP (11:45)

The technical core of the solution presented revolves around two distinct yet integrated platforms: No Intel's Griffin for comprehensive visibility and contextual intelligence, and Arcana AI for automated, deterministic decision-making.

No Intel's Griffin Platform: Unified Visibility with Generative AI

The Griffin SaaS platform is designed to provide a holistic view of an organization's attack surface. It aggregates information from various sources, including traditional vulnerability scanning, digital exposure monitoring, deep dark web threat intelligence, and GitHub repository surveillance. The goal is to unify these disparate data streams and transform them into actionable insights for security defenders.

A standout feature is its built-in chatbot, which allows users to interact with their organizational security data using natural language. This chatbot leverages two key agentic AI technologies:

1. Retrieval Augmented Generation (RAG):

• Purpose: To overcome the inherent context limitations of LLMs. LLMs are trained on vast datasets but lack specific, real-time organizational context.
• Mechanism: Relevant findings collected by Griffin (e.g., high CVSS scoring vulnerabilities, CVE information, dark web mentions) are vectorized and stored in a vector DB. When a user asks the LLM a question, the RAG system first retrieves the most relevant contextual information from this vector DB. This retrieved information is then augmented with the user's query and fed to the LLM, enabling it to generate a more accurate, relevant, and context-rich response.
• Data Ingestion: The decision of what data enters the RAG database is governed by static rules. For instance, if a vulnerability has a CVSS score of 9.8 or higher, or if it involves a confirmed SQL injection, that data is deemed significant enough to be vectorized and stored, preventing the RAG from being polluted with irrelevant information and ensuring focused conversations.

1. Multi-tool Coordination Protocol (MCP):

• Purpose: To empower LLMs to execute actions and interact with external systems.
• Mechanism: MCP allows LLMs to access and utilize a suite of tools, including running API queries. The Griffin architecture includes an MCP server and various MCP tools. When a structured query is posed, the LLM determines the appropriate API call to make, executes it, processes the raw data received, and then summarizes it for the user. This enables the chatbot to perform actions like "analyze asset data for a particular asset" by fetching real-time information from Griffin's backend.
• Workflow Visualization: The demo highlights the MCP workflow, showing steps like LLM tool selection, API query execution, data retrieval, data formatting, and LLM interpretation.

Arcana AI Platform: Deterministic Decision-Making with Deep Neural Networks

Arcana AI provides a distinct approach to automated decision-making, emphasizing reliability and control, especially for critical security operations.

1. Neural Network-Based Decision Models:

• Distinction from LLMs: Crucially, Arcana AI does not use LLMs for decision-making. Instead, it employs deep neural networks that are trained by subject matter experts within the organization. This ensures that the models reflect the company's specific context, policies, and risk appetite, avoiding the "internet bias" or non-deterministic nature of general-purpose LLMs.
• Training Process: Models are trained manually on historical data (e.g., six months of alerts) grouped by decision points. Experts label examples (e.g., true positive, false positive, specific remediation action). This iterative training continues until the model achieves a desired accuracy, typically over 90-95% on new, unseen scenarios. For instance, a model might be trained on 92 alerts to achieve 96% accuracy.
• Decision Options: These models can predict various courses of action, such as "remediate with blocking a firewall port," "creating a WAF rule," "trigger an investigation," or "archive the case."

1. Agentic Workflows for Automated Response:

• Triggering: Once Arcana's decision model makes a prediction, it triggers specific agentic workflows.
• Framework: These workflows are based on Google's agent development kit open-source framework and are essentially Python code. They define agents, assign them instructions, specify which model to use, and crucially, what MCP tools they should leverage.
• MCP Integration: Arcana's platform has its own MCP servers for various tools like Elastic, Splunk, Arcana itself, ServiceNow, and VirusTotal. In the demonstrated workflows, the No Intel MCP server is used to fetch additional context.
• Examples:
• Investigation: A dark web finding (e.g., exposed Jenkins console, planned exploit) triggers an investigation. Agents parse the asset, query No Intel via MCP for asset details and risks, confirm the vulnerability (e.g., Jenkins RCE, missing security headers, GitHub repo with secrets), and generate a detailed HTML report for human review.
• Automated Remediation: For clear-cut cases like a detected SQL injection vulnerability, the model might decide to automatically deploy a WAF rule to block SQL syntax in requests. For an exposed Docker API, it might trigger a request to block the relevant port.
• Archival: If a DDoS vulnerability advisory for Jenkins is received, but the instances are known to be for internal use only, the model can deterministically archive the alert as irrelevant, preventing alert fatigue and unnecessary human intervention.

This dual-platform approach provides both the broad visibility and contextual understanding needed to identify threats and the precise, automated decision-making capabilities required for efficient and effective security operations.

Demo / Proof of Concept

▶ Watch: Q&A: How Arcana AI models are trained for diverse scenarios (26:30)

The presentation included two distinct live demonstrations showcasing the capabilities of No Intel's Griffin platform and Arcana AI.

No Intel Griffin Demo

Catalin Neacsu first demonstrated the Griffin platform's chatbot functionality, highlighting its ability to interact with collected security data using RAG and MCP.

1. RAG-based Conversational Query:

• The demo started with a conversational query asking, "What is the biggest risk to my organization?"
• After processing, the chatbot, leveraging its RAG database, identified "API keys exposure" as the primary risk, providing an executive summary.
• A follow-up question, "Give me more information, which host exposed the API keys?" allowed the chatbot to retrieve specific details about the affected host, demonstrating its ability to maintain context and drill down into information.

1. MCP-based Structured Query:

• Switching to a "structured data" query mode, Catalin demonstrated how the MCP protocol enables direct interaction with Griffin's backend via API requests.
• A query like "analyze asset data for a particular asset" was submitted. The LLM first determined the correct API call, executed it, retrieved the raw data, and then summarized it.
• The workflow steps were visually presented, showing the LLM selecting the tool, running the API query, getting the data, formatting it, and interpreting it. This illustrated the deterministic execution of actions through MCP, contrasting with the conversational nature of RAG.

The demo emphasized Griffin's role as a "security partner" that allows users to have dynamic discussions with their own security data, gaining insights from various sources like vulnerability scans and dark web monitoring.

Arcana AI Demo

Darius Iakabos then presented the Arcana AI UI, focusing on the vulnerability management decision model.

1. Model Training and Accuracy:

• Darius explained that Arcana's models are trained manually by selecting relevant fields from alert bodies, labeling examples, and iteratively training until high accuracy is achieved.
• The demonstrated model was trained on approximately 92 alerts and achieved 96% accuracy on new findings. Its decision options included "remediate with blocking a firewall port," "creating a WAF rule," "trigger an investigation," or "archive the case."

1. Agentic Workflow Examples:

• Investigation Workflow:
• Scenario: A dark web finding from No Intel mentioned an enterprise IP, a public Jenkins console, and a plan to exploit it.
• Process: The workflow's first agent parsed the asset. It then used an MCP call to No Intel to cross-check assets and retrieve more information. This confirmed the presence of a Jenkins instance with an available console vulnerable to remote code execution (RCE), missing security headers, and a GitHub repository exposing environment variable files with secrets.
• Outcome: All findings were sent back to the decision model to update the case, and a final agent generated a detailed HTML report with findings, technical details, workflow steps, and recommendations, intended for automated delivery to relevant teams. This demonstrated how complex, multi-step investigations could be automated.

• Remediation Workflows:
• SQL Injection: A scanner detected an HTTP response indicating an SQL injection vulnerability. The decision model determined the best action was to add a WAF rule to block requests containing SQL syntax, a straightforward, automated response.
• Exposed Docker API: For a misconfiguration like an exposed Docker API, the model was trained to send a request to block the exposed port, preventing potential exploitation.

• Archival Workflow:
• Scenario: An advisory for a DDoS vulnerability in Jenkins was received.
• Process: The decision model, aware that the specific Jenkins instances were used only internally and DDoS was irrelevant in that context, decided to archive the case.
• Outcome: A notification was sent to stakeholders with the archival note, demonstrating the model's ability to make context-aware decisions that reduce alert noise.

The Arcana AI demo powerfully illustrated how custom-trained deep neural networks can make deterministic, accurate decisions, triggering automated agentic workflows to handle a wide range of security events from investigation to remediation and intelligent archival, significantly reducing the burden on human analysts.

Defensive Implications

▶ Watch: Q&A: Best LLMs for cybersecurity tasks and fine-tuning needs (35:00)

The insights and technologies presented in this talk offer several critical implications for security defenders looking to enhance their attack surface management capabilities:

1. Shift Beyond Scan-Only Vulnerability Assessment: Organizations must recognize that traditional vulnerability scanners provide only a partial view of their attack surface. A truly effective defense requires incorporating external threat intelligence from sources like the deep dark web, GitHub, and real-time campaign data to uncover exposures that exist outside the network perimeter.

1. Embrace Holistic Visibility: Implement platforms like No Intel's Griffin that unify data from diverse security domains. Gaining visibility into areas such as API keys exposure, leaked credentials, and misconfigurations on public-facing assets is crucial for identifying an attacker's potential entry points.

1. Leverage AI for Contextual Intelligence and Reporting: Utilize LLM-powered chatbots with RAG and MCP capabilities, as demonstrated by Griffin. These tools can help security teams quickly query their own data, gain deeper context, summarize complex findings, and generate custom reports, making information more accessible and actionable. However, understand that LLMs are best for data gathering and reporting, not for critical, deterministic decision-making.

1. Implement Deterministic AI for Decision-Making: Integrate machine learning-based decision models, such as those offered by Arcana AI, into security operations. These models, trained by subject matter experts on specific organizational data, can provide reliable and consistent triage and decision-making for security alerts. This helps overcome human capacity limitations and the non-deterministic nature of LLMs for critical tasks.

1. Automate Response Workflows: Design and deploy agentic workflows that are triggered by the decisions made by AI models. This allows for automated remediation actions, such as deploying WAF rules for SQL injection vulnerabilities, blocking exposed ports (e.g., Docker API), or initiating detailed investigations based on specific threat indicators. Automation frees up human analysts for more complex tasks.

1. Contextualize Risk and Prioritize Actions: Train decision models to understand the organizational context (e.g., whether a server is internal or internet-facing) to accurately assess the relevance and severity of vulnerabilities. This enables intelligent prioritization, allowing security teams to focus on true positives and critical threats while automatically archiving irrelevant alerts, thus reducing alert fatigue.

1. Maintain Human-in-the-Loop for Model Training and Oversight: While automation is key, human expertise remains vital for training the initial deep neural networks and continuously monitoring their performance. Security analysts will evolve into "model trainers" and "workflow orchestrators," ensuring the AI systems align with evolving security policies and threat landscapes.

By adopting these defensive implications, organizations can move from a reactive, scanner-dependent posture to a proactive, intelligent, and automated defense, significantly improving their ability to manage their dynamic attack surface and mitigate risks.

Key Takeaways

• Vulnerability assessment alone is insufficient: Traditional scanners miss critical exposures like leaked credentials and dark web mentions, which constitute a significant portion of an organization's true attack surface.
• Contextual intelligence is paramount: Integrating external threat intelligence (dark web, GitHub) with internal vulnerability data provides a holistic view, enabling defenders to see threats from an adversary's perspective.
• LLMs excel at data interaction, not critical decisions: Retrieval Augmented Generation (RAG) and Multi-tool Coordination Protocol (MCP) empower LLMs to provide contextual insights and execute data retrieval tasks, but their non-deterministic nature makes them unsuitable for critical security decision-making.
• Deep neural networks for reliable decision-making: Custom-trained deep neural networks (like Arcana AI's decision models) offer deterministic, accurate predictions for security alert triage and response, tailored to an organization's specific context and risk appetite.
• Automated agentic workflows for rapid response: Combining AI-driven decisions with agentic workflows enables automated actions, from deploying WAF rules and blocking ports to initiating complex investigations and generating detailed reports, significantly reducing response times.
• Human expertise shifts to model training and oversight: Security analysts transition from manual alert triaging to training and fine-tuning AI models and orchestrating automated workflows, maximizing efficiency and allowing focus on strategic security challenges.

About the Speaker(s)

Catalin Neacsu is a seasoned professional in the security industry with over 15 years of experience. His expertise spans security operations, threat intelligence, and forensics. More recently, Catalin has delved into agentic AI workflows and the application of AI for security monitoring and response, bringing this cutting-edge knowledge to the discussion.

Darius Iakabos serves as a Lead Solutions Architect at Arcana AI. His work involves designing and deploying decision models for security alerts, predictions, and agentic workflows at scale for various global companies. Darius's background is rooted in cyber security operations and automation, providing him with practical insights into leveraging AI for efficient and effective security solutions.