The Vulnerability Identity Crisis

The Vulnerability Identity Crisis

Speakers: Art Manion (Tharros Labs); Jay Jacobs (Empirical Security)

Conference: FIRSTCON

YouTube: https://www.youtube.com/watch?v=fmngLzUHphw

Overview

In "The Vulnerability Identity Crisis," Art Manion and Jay Jacobs present their ongoing work to establish a robust, structural, and foundational definition of a vulnerability. Their research, spanning several years and previous presentations at Voncon, highlights a critical challenge in the cybersecurity domain: the lack of a universally agreed-upon, practical definition for what constitutes a vulnerability. This ambiguity leads to pervasive disputes, hinders effective communication, and creates significant obstacles in the scalable management of vulnerability information across diverse contexts.

The talk delves into the shortcomings of existing vulnerability definitions, which often fail to account for complex "edge cases" or provide sufficient clarity for consistent application. Manion and Jacobs propose a novel rearchitecture of a vulnerability record, moving beyond simple dictionary definitions to a more comprehensive information model. This model aims to provide the necessary flexibility and precision to address the "identity crisis" that plagues vulnerability identification and management, ultimately seeking to ground the field in a shared understanding that can resolve long-standing debates.

Their work is not merely an academic exercise but a pragmatic effort to improve how security professionals identify, discuss, and mitigate vulnerabilities. By introducing a structural definition and an accompanying information architecture, they aim to facilitate better data exchange, enhance automated analysis, and clarify the roles of various stakeholders in the vulnerability disclosure and remediation lifecycle. The speakers emphasize that this is an evolving body of work, actively seeking feedback to ensure its practical applicability within the cybersecurity community.

Background

▶ Watch: The critical need for a structural vulnerability definition (1:20)

The genesis of this extensive research dates back to a fundamental question posed by Art Manion to Jay Jacobs in a pedestrian tunnel in the Netherlands: "What's the minimum amount of information that we need in a vulnerability record?" This inquiry quickly revealed that existing vulnerability identification systems, such as CVE (Common Vulnerabilities and Exposures), primarily function as naming conventions. While these systems provide identifiers, they often lack sufficient core required elements to unequivocally define the underlying vulnerability.

Early attempts by Manion and Jacobs focused on identifying five core elements for a "minimum viable vulnerability enumeration" (MVVE), which they presented at Voncon 2025 (as mentioned in the transcript). However, these initial frameworks proved inadequate, failing to address numerous edge cases and practical complexities. This led them to shift their focus from easy, archetypal vulnerabilities to those that are frequently disputed or poorly understood. Their investigation involved analyzing 29 different definitions from academic papers and frameworks, only to discover that none comprehensively covered these challenging edge scenarios.

Examples of these problematic edge cases abound. The "disputed" tag within the CVE database itself highlights numerous instances where consensus on a vulnerability's existence or scope is lacking. The Linux kernel community, for instance, often treats nearly every bug as a potential security vulnerability, sometimes leading to CVE assignments for issues that may never manifest as a security risk in a deployed system. Another illustrative example is the curl command-line utility. While a specific option (e.g., -L) combined with malicious input might allow code execution, it typically occurs under the user's own permissions, raising questions about its true security impact in a standard command-line scenario versus its potential exploitability within an automated pipeline or a poorly configured CGI application. Daniel Stenberg, the curl maintainer, notably dismisses such cases as "shooting yourself in the foot," underscoring the definitional divide. Furthermore, the way vulnerabilities are discussed and defined can vary significantly between different operational contexts, such as open-source libraries versus Operational Technology (OT), SCADA, or ICS environments.

This pervasive lack of a clear, shared definition underpins what Manion and Jacobs term the "Vulnerability Identity Crisis." They cite a core principle from their prior work: "Unsettled terms produce disputes that no argument can resolve." This principle directly addresses the frustration seen in debates, such as those concerning the application of CVSS scores to unproven Linux kernel bugs, where differing interpretations of "vulnerability" lead to unproductive arguments rather than resolution. The speakers contend that without a foundational, structural definition, the industry will continue to struggle with inconsistent reporting, ineffective mitigation strategies, and an inability to scale vulnerability management efforts effectively.

Key Findings

▶ Watch: Edge cases: Curl and Linux kernel demonstrate definition problems (6:00)

The central premise of "The Vulnerability Identity Crisis" is that existing vulnerability definitions are structurally flawed, leading to ambiguity and disputes. Manion and Jacobs assert that a vulnerability is not merely a "weakness" but a more complex construct, which they define as a disposition. This concept is foundational to their proposed structural redefinition.

Their analysis of 29 different vulnerability definitions revealed common elements, such as the mention of a "weakness" (often linked to CWE), a "system" or "bearer" (the affected entity), and probabilistic statements like "can be exploited." However, a critical deficiency identified was the inconsistent or absent inclusion of "consequence" or "outcome" in many definitions. For instance, while CV.org's definition includes consequence, NIST SP800-30 and the EU Cyber Resilience Act (CRA) do not explicitly detail the outcome of an exploitation. This omission leaves crucial questions unanswered, such as "exploited for what?"

To address these shortcomings, the speakers introduce a four-part structural definition that must be met for a vulnerability to exist, moving beyond a simple linguistic definition to one with essential information aspects:

1. System: The specific hardware and/or software component that is affected.
2. Fault: The underlying flaw, bug, or weakness within the system.
3. Conditions: The specific circumstances under which the fault will manifest as a security issue.
4. Security Failure: The undesirable outcome where a security property is violated, resulting in a net gain for an attacker and a net loss for the defender.

The culmination of these four elements defines a disposition. Drawing from philosophy, particularly the work of Gilbert Ryle, Manion and Jacobs use the analogy of glass having a disposition to shatter. The glass is not shattered until specific conditions (e.g., sufficient force in a particular area) are met, at which point the disposition manifests. Similarly, exploitable code has a disposition to be exploited, but only when the defined system, fault, and conditions align to cause a security failure. This holistic view provides a clear, actionable framework for consistently identifying and communicating vulnerabilities, thereby resolving the "identity crisis" by establishing what information must be present to claim a vulnerability exists.

Technical Deep Dive

▶ Watch: Unifying concept: A vulnerability is a disposition (20:00)

The core of Manion and Jacobs' structural definition rests on four interconnected elements: the System, the Fault, the Conditions, and the Security Failure. Each component is rigorously defined to provide clarity and facilitate consistent identification and communication of vulnerabilities.

The System, or "bearer," refers to the affected entity. This isn't just software; it's a computing system, encompassing both hardware and software, as "software doesn't run by itself [and] hardware doesn't do a whole lot without software." In practical terms, this could be a "product" (as in CVE terminology), a binary, source code, a script, an application, or any "unit of the system that is the thing that is affected." Crucially, the system itself is not specific to a vulnerability; it can exist independently.

The Fault is the underlying flaw, bug, or weakness. While CWE (Common Weakness Enumeration) provides a practical connection, the fault itself is not the vulnerability. It's the "possibility of it later being exploited," representing "unwanted unintended behavior" or a "coding mistake," such as a buffer overflow or a lack of authentication. Like the system, a fault can exist without necessarily constituting a vulnerability until other elements are present.

The most challenging and underdeveloped aspect, according to the speakers, lies in defining the Conditions. These are the "circumstances under which this will manifest" and "have to be true" for a vulnerability to be exploitable. The current language around conditions, particularly in frameworks like CVSS, is described as "barbaric," lacking the precision needed for effective communication. Conditions can be categorized into actor conditions (e.g., creating a specific packet, delivering an exploit) and environmental conditions (e.g., system configuration). The talk highlights Log4Shell as an example, noting that 16 distinct conditions had to be met for its exploitation. The critical insight here is that disabling or mitigating any single condition can prevent the attack from succeeding. While concepts like "user interaction" or "attack complexity" in CVSS touch upon conditions, a strict, comprehensive terminology is still missing.

Finally, a Security Failure must occur. This means a "security boundary is crossed," resulting in a "net gain to the attacker, a net loss to the defender or the target." The infamous curl -L example is revisited: if code execution occurs within the user's existing permissions, there is no security impact because no boundary is crossed. Similarly, reports starting with "once you have administrative privileges" often describe actions that don't constitute a security failure if the attacker already possesses root-level access. Manion and Jacobs critique the use of CIA (Confidentiality, Integrity, Availability) triads for describing security failures, arguing they are too abstract. Instead, they advocate for a more explicit model based on Principal, Action, and Object (PAO). This model describes who (a principal, identity, or role) performs what (an action) on what (an object), potentially under specific conditions. This level of detail clarifies the exact authorization violation or unintended behavior, providing a far more precise understanding of the security impact.

The culmination of these four elements defines a disposition. A vulnerability is not a static weakness but an inherent tendency of a system, characterized by a fault, to produce a security failure under specific conditions. This philosophical concept, analogous to glass having a disposition to shatter, emphasizes that the vulnerability exists as a potential, but only manifests when all conditions are met. Therefore, a complete vulnerability record must enumerate the system, the fault, the conditions, and the resultant security failure to fully describe its disposition.

This structural definition underpins a proposed rearchitecture of vulnerability records, famously illustrated by the "peg and bags" analogy. A vulnerability identifier (like a CVE ID) is merely a "peg"—a name—and not a description. As borrowing from John Searle, it's "a peg on which to hang descriptions." These descriptions are "bags," or assertions, each independently supporting one or more sub-parts of the structural definition (system, fault, conditions, security failure). For a vulnerability claim to be valid, enough "bags" must be present to prove all four elements of the disposition are met. This model offers several advantages: it separates the identifier from its description (preventing "description baked into the ID recipe for long-term failure"), allows for dynamic evolution of information (bags can be added, disputed, or removed over time), and provides provenance (who asserted what, when, and why). This architecture directly supports initiatives like the proposed CDP (Coordinated Disclosure Program) for CVE, enabling broader community contributions and more complete, reliable vulnerability records.

Demo / Proof of Concept

▶ Watch: Peg and Bags model for flexible vulnerability records (25:30)

While the talk did not feature a live demonstration or a proof-of-concept tool, the speakers presented a robust conceptual framework and a "technical drawing" illustrating the relationship between the proposed elements of System, Fault, Conditions, and Security Failure, all culminating in the idea of a Disposition. The emphasis was on the theoretical rearchitecture of vulnerability information and the philosophical underpinnings of their proposed definition, rather than a practical, executable demonstration. The speakers indicated that testing their proposed model on "undecided records" is a next step for their ongoing work.

Defensive Implications

▶ Watch: Q&A: The philosophical challenge of precise security terminology (34:40)

The structural definition of a vulnerability proposed by Art Manion and Jay Jacobs offers profound implications for cybersecurity defenders, moving beyond current ambiguities to foster clearer communication, more effective mitigation, and improved risk management.

Firstly, the emphasis on a precise, four-part definition (System, Fault, Conditions, Security Failure) culminating in a disposition provides defenders with a consistent framework for evaluating potential threats. This clarity helps to reduce the "identity crisis" where a bug's status as a vulnerability is debated, allowing teams to quickly ascertain whether all necessary criteria are met. This will streamline initial triage and reduce wasted effort on issues that do not constitute a true security vulnerability under the defined terms.

Secondly, the detailed focus on Conditions is a game-changer for defensive strategies. By explicitly enumerating the "circumstances under which this will manifest," defenders can identify specific environmental configurations, actor prerequisites, or system states that must be true for an exploit to succeed. This granular understanding empowers organizations to implement highly targeted compensating controls. As highlighted with the Log4Shell example, if any one of the 16 necessary conditions can be disabled or altered, the attack chain is broken. This shifts mitigation from a broad-brush approach to precise, condition-based defenses, potentially preventing exploitation even if the underlying fault remains.

Thirdly, the critique of generic security properties like CIA (Confidentiality, Integrity, Availability) and the advocacy for Principal, Action, Object (PAO) for describing Security Failures provides a more actionable language for defenders. Instead of merely knowing that "integrity" is compromised, understanding which principal performed what action on which object allows for more precise authorization controls, logging, and incident response. This specificity helps defenders configure systems to prevent unauthorized actions at a much finer grain, directly addressing the root cause of the security failure.

Furthermore, the "peg and bags" model for vulnerability records significantly enhances vulnerability management. By treating a vulnerability identifier as a mere "peg" for independent "assertions" (bags) that describe the system, fault, conditions, and security failure, defenders gain a dynamic and transparent view of vulnerability information. This model allows for:

• Evolving Information: As new details emerge post-publication (e.g., specific exploit conditions, affected versions), they can be added as new assertions with clear provenance (who said what, when).
• Trust and Provenance: Knowing who made which assertion (e.g., "Jay on this date said...") allows defenders to weigh the credibility of information sources, which is crucial in a complex threat landscape.
• Customization: Organizations can choose to collect and prioritize assertions that are most relevant to their specific risk profile and operational context, avoiding the "Swiss army knife" approach of ever-growing, unwieldy vulnerability records.

Finally, the framework's ability to accommodate "insecure by default" configurations and protocol vulnerabilities (like outdated cryptography such as MD5) under the same structural definition is vital. It acknowledges that a "fault" isn't exclusively a coding bug but can be a misconfiguration or an outdated design choice, and "conditions" can include external factors (e.g., advances in cryptanalysis). This broadens the scope of what defenders can systematically identify and manage as vulnerabilities, ensuring a more comprehensive security posture.

Key Takeaways

• Current vulnerability definitions are inconsistent and inadequate, leading to persistent disputes and hindering effective communication and scalable management of security information.
• A vulnerability should be understood as a disposition, which is a combination of a specific System, an underlying Fault, precise Conditions for manifestation, and a resulting Security Failure.
• The "Conditions" component is critically underdeveloped in current terminology (e.g., CVSS's "barbaric" language), yet it is essential for identifying actionable mitigation strategies and breaking attack chains.
• Security failures should be described in terms of Principal, Action, Object (PAO) rather than abstract concepts like Confidentiality, Integrity, and Availability (CIA) to provide greater clarity and enable more precise defensive measures.
• Vulnerability identifiers (like CVE IDs) are merely "pegs" for hanging independent "bags" of assertions, which describe the vulnerability's structural elements and include provenance (who made the assertion, when).
• This structural rearchitecture of vulnerability information aims to improve clarity, enable more targeted defensive strategies, enhance vulnerability management processes, and resolve long-standing debates within the cybersecurity community.

About the Speaker(s)

Art Manion is associated with Tharros Labs and is deeply involved in the CVE program, serving as a board member and chairing one of its working groups. His work focuses on the foundational aspects of vulnerability information and definitions, aiming to improve the structural integrity and utility of vulnerability records.

Jay Jacobs is affiliated with Empirical Security and also plays a significant role in the CVE program, chairing the CVE Consumer Working Group. His expertise lies in understanding how vulnerability information is consumed and applied by various stakeholders, driving efforts to make these definitions more practical and actionable for the broader security community.

Both speakers have been collaborating for several years on this ongoing research, presenting their evolving ideas at conferences like Voncon, and are committed to refining the understanding and management of vulnerabilities.