Deriving CVSS from Multi-Scenario Attack Graphs: A Reproducible, Auditable Scoring Method

Deriving CVSS from Multi-Scenario Attack Graphs: A Reproducible, Auditable Scoring Method

Speakers: Karel Knibbe (Volerion); Ruben Bos (Volerion)

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=Qdu2i8gTpWQ

Overview

In an era where the volume of published vulnerabilities is skyrocketing—with projections from First Forecast estimating 100,000 new vulnerabilities this year alone—the traditional, manual approach to Common Vulnerability Scoring System (CVSS) analysis is no longer sustainable. Ruben Bos and Karel Knibbe of Volerion addressed this critical challenge at VulnCon, presenting an innovative, graph-based methodology designed to transform CVSS scoring from a subjective, inconsistent exercise into a reproducible, auditable, and automatable process. Their talk dissects the inherent flaws in current CVSS practices, particularly the propensity for disparate scores from different analysts, and introduces a robust framework to overcome these limitations.

The core of their presentation revolves around the concept of explicitly modeling attack scenarios rather than attempting to score a vulnerability in isolation. This paradigm shift aims to eliminate the "Frankenstein factors" that arise when analysts inadvertently blend characteristics from multiple attack paths, leading to scores that misrepresent true risk. By building granular attack graphs that detail every atomic action, condition, and impact, Volerion's approach offers an unprecedented level of transparency and consistency, empowering defenders to prioritize threats with greater accuracy and speed.

This talk is particularly pertinent for security professionals grappling with vulnerability overload, those seeking to integrate automation into their risk assessment workflows, and anyone striving for more objective and reliable vulnerability prioritization. The proposed methodology not only promises to streamline CVSS calculation but also lays the groundwork for a single source of truth that can inform broader security conclusions, such as attack chaining and automatebility metrics.

Background

▶ Watch: Introduction: The problem of subjective CVSS scoring (0:00)

The pervasive perception that CVSS scoring is subjective is a significant impediment to effective vulnerability management, especially when attempting to scale operations through automation. As Ruben Bos highlighted, when two different analysts score the same vulnerability, they often arrive at divergent factors, undermining confidence and consistency. This issue is not merely academic; it directly impacts an organization's ability to accurately prioritize and respond to threats in a timely manner.

Volerion's journey to address this challenge began with attempts to automate CVSS derivation using artificial intelligence. Their initial approach involved "one-shot" prompts to a model, asking for a CVSS factor for a given vulnerability. While the model produced plausible-looking factors, these results were highly inconsistent, changing with each recalculation, and offered no insight into the AI's reasoning. This lack of audibility rendered the approach unusable for reliable customer delivery.

A subsequent attempt involved a "walk-through" approach, steering the AI through a series of questions for each CVSS metric. This improved consistency and provided some insight into the AI's thought process. However, a critical flaw emerged: the AI frequently mixed attack scenarios, answering questions based on different, sometimes contradictory, attack paths. For instance, it might score an attack as remote while simultaneously assuming no privileges, despite a remote attack typically requiring initial privileges. This led to what Volerion termed "Frankenstein factors"—cherry-picked metrics from disparate scenarios that did not accurately represent any single, coherent attack path.

Through extensive analysis, Volerion identified six primary reasons why CVSS factors diverge, underscoring the deep-seated issues with current scoring practices:

1. Subjective Interpretation: The CVSS specification's guidance to "score on the reasonable worst-case scenario" leaves room for subjective interpretation of "reasonable," leading to different conclusions about potential impacts (e.g., denial of service vs. code execution for a memory bug).
2. Varying Knowledge: Analysts possess different levels of knowledge or access to information about a vulnerability, causing discrepancies in their assessment of required bypasses or conditions.
3. Specification Misinterpretation: The complexity of the CVSS specification itself can lead analysts, even in good faith, to misinterpret metrics like "subsequent impact" versus "vulnerable system."
4. Intentional Manipulation: In some cases, individuals may intentionally manipulate scores to achieve specific outcomes, such as higher bug bounties, hoping the discrepancy goes unnoticed.
5. Forgetting Requirements: When scoring a mental model of a vulnerability rather than an explicit attack scenario, analysts can easily overlook critical requirements, such as user interaction, leading to incomplete or inaccurate assessments.
6. Mixing Scenarios: The most problematic issue, as observed in their AI experiments, is the blending of characteristics from multiple distinct attack scenarios into a single, composite CVSS factor that truthfully represents none of them.

These challenges highlight a fundamental flaw: attempting to score the vulnerability itself rather than the specific, explicit attack scenarios it enables. The core tenet of Volerion's proposed solution is therefore simple yet profound: score the scenario, not the vulnerability. By making attack scenarios explicit and agreeing upon them before assigning CVSS labels, much of the subjectivity and inconsistency can be eliminated, paving the way for truly reproducible and auditable vulnerability scoring.

Key Findings

▶ Watch: Six reasons why CVSS factors diverge (6:00)

The central discovery and contribution of Volerion's work is the development and validation of a graph-based approach for deriving CVSS scores. This methodology fundamentally shifts the focus from subjective interpretation of vulnerability characteristics to objective modeling of explicit attack scenarios, yielding reproducible and auditable results.

The key findings and contributions include:

• Scenario-Centric Scoring: The most crucial insight is that reliable CVSS scoring necessitates modeling individual attack scenarios rather than attempting to score a vulnerability in abstract. This directly addresses the "Frankenstein factors" problem by ensuring that all derived metrics pertain to a single, coherent sequence of actions.
• Reproducibility through Explicitness: By representing each atomic action, condition, and impact within an attack scenario as nodes and edges in a graph, every characteristic becomes explicit. This eliminates ambiguity and ensures that different analysts, or automated systems, can consistently arrive at the same score for a given scenario.
• Auditable Derivation: The graph structure provides inherent audibility. Every metric value (e.g., Attack Vector, Privileges Required, User Interaction) can be traced back to specific properties and relationships within the graph, allowing for transparent verification of the scoring logic.
• Comprehensive Scenario Evaluation: The approach facilitates the modeling and evaluation of all plausible attack scenarios associated with a vulnerability. This is critical because, as demonstrated in their examples, the highest-scoring scenario is not always immediately obvious and can be missed if only one is considered or guessed.
• Objective Attack Complexity: Volerion successfully deconstructed the often-subjective Attack Complexity (AC) metric (and Attack Requirements (AT) in CVSS 4.0) into a fixed set of objective condition types. This innovation provides a standardized way to assess the effort an attacker must expend, moving beyond vague interpretations of "complexity."
• Single Source of Truth: The constructed attack graph transcends its utility for CVSS. It serves as a single source of truth for a multitude of security analyses, including deriving automatebility metrics, mapping to CAPEC (Common Attack Pattern Enumeration and Classification) entries, and identifying potential for attack chaining by understanding privilege gains and system access.

In essence, Volerion's findings demonstrate that by embracing a structured, graph-based representation of attack dynamics, the long-standing challenges of CVSS subjectivity and inconsistency can be overcome, paving the way for more accurate, reliable, and scalable vulnerability management.

Technical Deep Dive

▶ Watch: Step-by-step: Building the graph and deriving CVSS metrics (11:00)

Volerion's graph-based methodology for CVSS derivation is rooted in a structured, programmatic approach to modeling attack scenarios. The process begins with constructing an attack graph that captures the atomic actions involved in an exploit, focusing exclusively on the delivery and exploitation phases, as these are most relevant to CVSS. Persistence and preparation phases are intentionally excluded to keep the scope aligned with CVSS definitions.

The graph is built from a top-to-bottom perspective, where nodes represent individual actions and edges define the flow between them. Each action within the graph is attributed to one of three types of actors: an attacker, a user (which can be a victim), or a system. This distinction is fundamental for deriving various CVSS metrics.

The entire graph, including its nodes, edges, and properties, is represented in JSON format. This programmatic representation is key to achieving automation and audibility. Each node in the JSON array corresponds to an action and can possess several properties that characterize it:

• Proximity: For attacker or user actions, proximity (e.g., remote, local, adjacent, physical) is defined relative to the vulnerable system. This directly informs the Attack Vector (AV) metric.
• Phase: Each action is categorized into either the delivery or exploitation phase. This, in conjunction with other properties, helps determine the Attack Vector.
• User Interaction (UI): This metric is derived programmatically by simply checking for the presence of a "user" actor within the attack path. If a user is required to perform an action to facilitate the attack, then User Interaction is present.
• Privileges Required (PR): Unlike a global attribute, privileges are attached to the specific action where they are needed. If any action along the attack path requires privileges, the overall Privileges Required metric for that scenario is adjusted accordingly.
• Attack Complexity (AC) / Attack Requirements (AT): This is one of the most significant technical advancements. Instead of subjective assessment, complexity is modeled through conditions attached to specific paths or actions within the graph. These conditions are categorized into a fixed set of types, which Volerion developed after extensive analysis of CVSS 3.1 and 4.0 specifications. Examples of condition types include:
• Configuration: A specific system configuration must be present.
• Secret: A secret (e.g., a token, password) must be known or obtained.
• Bypass: A security control must be bypassed.
• Position: The attacker must be in a specific physical or logical position.
• Timing: The attack must occur within a specific time window or sequence.
• Interaction: Specific user interaction beyond simply triggering the vulnerable component.

By categorizing these conditions, the assessment of attack complexity becomes objective and quantifiable, directly influencing the AC (CVSS 3.1) or AT (CVSS 4.0) metric.

• Impact (CIA): Impact, referred to as "effect" in Volerion's model, is attached to specific actions within the graph. An action can yield effects on Confidentiality (C), Integrity (I), and Availability (A). This granular attachment ensures that impacts are contextually linked to the exploit steps.
• Scope: The Scope metric is derived from the systems impacted by the effects. Systems are defined globally within the JSON structure and can be of type "vulnerable" (the system with the flaw) or "subsequent" (another system affected by the attack). If an effect impacts a subsequent system, a Scope Change is indicated.

Once the graph is fully constructed with all relevant properties, the CVSS metrics for a given scenario are programmatically derived by traversing the graph and aggregating the characteristic values. This systematic derivation, rather than a checklist-based approach, ensures that the resulting CVSS vector is a true reflection of the modeled attack scenario. The JSON representation of a node, for instance, would include properties like proximity, phase, effects (which in turn point to systems with name and type), and edges linking to subsequent actions, providing a complete and auditable record of the attack path.

Demo / Proof of Concept

▶ Watch: Complex example: Grafana with two distinct attack paths (21:40)

Volerion illustrated the power of their graph-based approach through several compelling examples, contrasting public CVSS scores with those derived from their explicit scenario modeling. These demonstrations effectively highlighted how subjective interpretations lead to inaccurate scores and how a structured approach uncovers critical details.

Example 1: Desktop Application Crash Report Vulnerability

The first example involved a desktop application vulnerability where submitting a crash report accidentally included the user's Wi-Fi password. The public CVSS record for this vulnerability stated "no complexity" and "no user interaction," affecting both the vulnerable and a subsequent system in terms of confidentiality.

Volerion's graph modeling revealed a different picture:

• Condition: A specific execution condition was required—a failure had to occur during installation. This explicitly introduces an element of Attack Complexity.
• User Interaction: The user needed to actively submit the crash report, clearly indicating User Interaction.

The graph demonstrated that the public factor was incorrect, having missed crucial details regarding both complexity and user involvement.

Example 2: PDF Reader Code Execution

This example featured a PDF reader vulnerable to code execution via malicious PDFs, presenting two distinct attack scenarios:

1. A victim locally imports a malicious PDF.
2. An attacker tricks a victim into visiting a malicious website, leveraging a browser plugin to import the PDF.

The public CVSS for this vulnerability suggested a network attack vector with no complexity.

Volerion's analysis, however, modeled both scenarios explicitly:

• Scenario 1 (Local Import): This path involved a Local Attack Vector and potentially active user interaction.
• Scenario 2 (Browser Plugin): This path represented a Network Attack Vector with an Attack Requirement (the plugin being installed and enabled).

While both scenarios were similar in overall severity, the graph clearly distinguished their characteristics. Crucially, by modeling both, Volerion could definitively identify the higher-scoring scenario (the browser plugin one, with a network AV and AT present), preventing analysts from guessing or overlooking the most severe path. This directly addressed the problem of "Frankenstein factors" by ensuring each score was derived from a singular, consistent attack chain.

Example 3: Grafana Open Redirect (CVE-2021-39226)

The final, most illustrative example involved a Grafana vulnerability (CVE-2021-39226) that allowed for either code execution via an open redirect (if Content Security Policy, CSP, was disabled) or Server-Side Request Forgery (SSRF) if the Grafana image renderer plugin was installed. The public CVE factor indicated "user interaction required" and "no privileges," suggesting it was scored based on the less severe cross-site scripting (XSS) scenario.

Volerion's graph revealed two entirely distinct attack paths that did not merge:

• Path 1 (XSS): This path had a critical condition: the CSP needed to be bypassed or disabled for JavaScript execution in the browser. Using CVSS 4.0, this scenario derived a "Low" severity score.
• Path 2 (SSRF): This path also had a condition: the image renderer plugin needed to be installed to gain access to internal network systems. This scenario, when derived using CVSS 4.0, resulted in a "Medium" severity score.

This demonstration powerfully illustrated that by explicitly modeling both scenarios, Volerion's approach correctly identified the SSRF as the more severe attack path, a detail missed by the public CVSS factor that likely focused solely on the XSS. The ability to visualize and compare these distinct paths ensures that the most impactful scenario is always considered, preventing underscoring due to incomplete analysis.

Across these examples, the core message was consistent: explicit, graph-based modeling eliminates subjectivity, uncovers hidden attack requirements, and ensures that the highest-severity scenario is accurately represented, leading to more reliable and actionable CVSS scores.

Defensive Implications

▶ Watch: Q&A: Improving attack complexity granularity in CVSS (28:00)

Volerion's graph-based approach to CVSS derivation offers profound implications for defenders, providing a more robust, reproducible, and actionable framework for vulnerability management.

Firstly, the method directly addresses the critical need for accurate prioritization. With the sheer volume of vulnerabilities being disclosed, defenders are overwhelmed. By providing auditable and consistent CVSS scores, derived from explicit attack scenarios, organizations can confidently allocate resources to the most critical threats. This eliminates the uncertainty and debate often associated with subjective CVSS assessments, allowing security teams to focus their efforts where they will have the greatest impact.

Secondly, the detailed attack graphs serve as an invaluable source of intelligence for understanding threats. Instead of a generic CVSS score, defenders gain a clear, step-by-step blueprint of how an attack can unfold. This explicit modeling includes specific conditions, such as the requirement for a particular plugin to be installed or a CSP to be disabled. This level of detail empowers defenders to:

• Identify specific mitigation points: Knowing the exact conditions and actions required for an exploit allows for targeted defensive measures.
• Improve threat intelligence: The graphs provide a deeper understanding of attack prerequisites and consequences, enhancing an organization's overall threat intelligence posture.
• Tailor responses: Defenders can analyze which scenarios are most relevant to their specific environment and tailor their response strategies accordingly, rather than relying on generic assessments.

Thirdly, the inherent automatable nature of the graph-based methodology is a game-changer for scaling vulnerability management. As manual analysis becomes increasingly unfeasible, the ability to programmatically derive CVSS scores ensures that defenders can keep pace with the ever-growing number of vulnerabilities. This speed and consistency are vital for responding to fast-evolving threats and maintaining a strong security posture.

Moreover, the reproducibility of scores eliminates discrepancies between different analysts or systems. This consistency is crucial for internal risk assessment, compliance reporting, and communication with external stakeholders. Everyone operates from the same, verifiable understanding of a vulnerability's severity. The ability to model all plausible scenarios also enables environment-specific scoring, allowing organizations to determine the highest severity scenario relevant to their unique infrastructure and configurations, rather than being bound by a single, generic public score.

Finally, the attack graph itself is presented as a single source of truth with utility extending far beyond CVSS. It can inform a multitude of other security analyses:

• Automatebility: The explicit steps, complexities, and actor proximities within the graph can be used to derive metrics related to how easily an attack can be automated.
• CAPEC Mapping: The atomic actions and conditions within the graph can be mapped to entries in the Common Attack Pattern Enumeration and Classification (CAPEC), enriching threat understanding.
• Attack Chaining: By identifying the starting and ending points of attack paths, along with privileges required and gained, the graph can facilitate the analysis of potential attack chains, revealing how multiple vulnerabilities might be exploited in sequence to achieve broader objectives.

In summary, Volerion's approach equips defenders with the tools to move beyond subjective guesswork, enabling precise prioritization, detailed threat understanding, scalable automation, and a unified, auditable framework for comprehensive vulnerability assessment.

Key Takeaways

• Score the Scenario, Not the Vulnerability: To achieve accurate and reproducible CVSS scores, focus on explicitly modeling individual attack scenarios rather than attempting to assess a vulnerability in isolation. This eliminates subjective interpretation and "Frankenstein factors."
• Model All Plausible Scenarios: Do not guess which attack path is the most severe. Explicitly model every plausible scenario associated with a vulnerability to ensure the highest-scoring and most impactful threat is identified and prioritized.
• Derive Metrics Programmatically: Shift from a checklist-based CVSS assessment to programmatically deriving metric values from explicit attack characteristics within a graph. This enhances accuracy and consistency compared to manual, subjective evaluations.
• Auditable Graphs for Reproducibility: Attack graphs provide inherent audibility, showing precisely why each metric value was chosen based on specific properties and relationships. This transparency is fundamental for achieving reproducible and trustworthy vulnerability scores.
• Attack Graphs as a Single Source of Truth: The detailed attack graph is a versatile asset, serving as a foundational data structure for various security analyses beyond CVSS, including automatebility metrics, CAPEC mapping, and identifying opportunities for attack chaining.
• Enhance CVSS Specification for Objectivity: Future CVSS versions could benefit from more granular and objective definitions for metrics like Attack Complexity/Requirements, differentiating between various condition types and their actual effort to overcome, rather than bundling them broadly.

About the Speaker(s)

Ruben Bos and Karel Knibbe are security researchers and entrepreneurs from Volerion. Both come from a strong background in offensive security, with extensive experience in red teaming, bug bounty hunting, and penetration testing. Their journey into automating vulnerability assessment began in 2019 with the automation of bug bounty hunting. Recognizing the critical need for scalable and accurate vulnerability data, they further ventured into automating CVE scanning in 2022. This led them to found Fluirion in 2022, a company dedicated to solving the pervasive problems of delayed, inaccurate, and inconsistent public CVE data through large-scale CVE enrichment. Their work at Volerion and Fluirion reflects their commitment to transforming subjective security challenges into objective, scalable, and automatable solutions for the benefit of defenders.