Vulnrichment: Year One

Vulnrichment: Year One

Speakers: Art, Lindsay (CISA)

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=g5pSVMnWD7k

Overview

This talk, "Vulnrichment: Year One," delves into the U.S. Cybersecurity and Infrastructure Security Agency's (CISA) initiative to enhance the utility and completeness of Common Vulnerabilities and Exposures (CVE) records. Presented by Art and Lindsay from CISA, the session reflects on the first year of the Vulnrichment program, a critical effort to democratize CISA's extensive internal vulnerability analysis and provide it as a public service. The program aims to bridge data gaps in CVEs, offering richer context and actionable intelligence for vulnerability management practitioners.

The significance of Vulnrichment lies in its direct impact on improving the security posture of federal civilian executive branch (FEB) agencies, state, local, tribal, and territorial (SLTT) governments, and critical infrastructure partners. By systematically adding valuable data points such as CVSS (Common Vulnerability Scoring System) scores, CWE (Common Weakness Enumeration) identifiers, and references, Vulnrichment empowers organizations to prioritize and respond to vulnerabilities more effectively. The speakers highlight the program's commitment to transparency, community engagement, and continuous experimentation as foundational principles for its ongoing evolution.

Through a candid review of its operational successes and challenges, including an insightful experiment with CPE (Common Platform Enumeration) enrichment, the talk underscores CISA's dedication to evolving vulnerability management practices. It serves as a testament to the agency's role as an operational lead in vulnerability response, striving to make security information more accessible and impactful for the broader cybersecurity community, ultimately contributing to a more secure digital ecosystem.

Background

▶ Watch: Welcome and talk introduction (0:00)

CISA's mission is expansive: to reduce the prevalence and impact of vulnerabilities across the federal enterprise, encompassing FEB agencies, SLTT governments, and a diverse array of public and private sector critical infrastructure partners. This broad mandate means CISA's work, including its vulnerability analysis, often influences organizations beyond these core spheres. Over the past 10 to 15 years, CISA has honed its approach, leveraging its authorities and expertise to address current operational states while also advocating for secure by design and default technology.

The agency has a long history of deep vulnerability analysis, conducting Coordinated Vulnerability Disclosure (CVD) for nearly 15 years, with its first official CVD advisory dating back to the 2010-2012 timeframe. This extensive experience has cultivated significant in-house expertise. Beyond CVD, CISA utilizes several key frameworks and products to execute its mission:

• Stakeholder Specific Vulnerability Categorization (SSVC): CISA scores every new CVE using SSVC to prioritize its response to vulnerabilities affecting FEB agencies. This process requires substantial analytical expertise.
• Known Exploited Vulnerabilities (KEV) Catalog: A widely used resource, the KEV catalog identifies vulnerabilities that are actively being exploited. Federal civilian executive branch agencies are mandated to address KEV vulnerabilities through Binding Operational Directive (BOD 22-01).
• Common Security Advisory Framework (CSAF): Since 2022, CISA has been providing its vulnerability advisories in this machine-readable format, facilitating automated consumption of vulnerability information.

Prior to conceptualizing Vulnrichment, CISA had an inherent goal of democratizing the vast amount of analysis produced internally for its operational mission. The idea was to take all this information, synthesize it, and make it publicly available in an easily consumable format. This ethos laid the groundwork for Vulnrichment, which was designed with several core goals:

1. Providing Useful Data: To share the valuable vulnerability triage and analysis data CISA was already producing internally.
2. Leveraging Existing Capabilities: To avoid reinventing the wheel by feeding CISA's internal analysis into the CVE ecosystem through the CVE Program's Authorized Data Publisher (ADP) capability.
3. Transparency: To maintain an open process using a public GitHub repository for documentation, issue resolution, and change requests.
4. Accuracy (within scope): To fill data gaps without overwriting or auditing information provided by the originating CNA (CVE Numbering Authority). Vulnrichment's role is to enrich, not to engage in protracted debates over scores or classifications.

Key Findings

▶ Watch: CISA's long history in vulnerability analysis (4:00)

The "Year One" review of Vulnrichment presented several significant findings and lessons learned, offering valuable insights into the complexities of large-scale vulnerability data enrichment:

• Increased CNA-Provided Enrichment: The talk observed a notable trend where the number of CVEs enriched by the originating CNA began to surpass those enriched by Vulnrichment. This shift, illustrated by a crossing of lines in their internal metrics, is tentatively attributed to the CVE Program's increased advocacy for CNAs to provide comprehensive data (like CVSS and CWE) directly when creating CVE records. This suggests a positive impact of encouraging upstream data provision.
• Proactive Community Engagement and Responsiveness: The public nature of the Vulnrichment GitHub repository proved to be a powerful mechanism for community engagement. The team highlighted instances of rapid response to public feedback, such as fixing a Curl-related issue within hours on a Sunday after a researcher raised a concern via Mastodon and a GitHub issue. This demonstrates the effectiveness of transparency in fostering a collaborative environment for data improvement.
• Challenges and Experimentation with CPE: A significant finding was the experience with CPE enrichment, conducted as a "committed sincere experiment" from April to December 2024. The initiative found that while syntactically correct CPEs are easy to generate, ensuring semantic correctness and adherence to the NVD (National Vulnerability Database) dictionary was exceptionally difficult and time-consuming. This effort consumed approximately two-thirds of the team's record processing time, ultimately leading to the decision to cease CPE enrichment. The speakers concluded that the problem of "globally naming software" remains fundamentally unsolved and poses a significant hurdle to effective vulnerability management.
• Value of Non-Authoritative Enrichment as a Testbed: Vulnrichment served as a valuable experimentation ground, even when initiatives like CPE enrichment didn't yield the desired long-term outcomes. This hands-on experience manipulating data provided operational lessons that can inform future recommendations for addressing ecosystem-wide challenges. The speakers emphasized that non-authoritative enrichment, while useful for filling gaps, ultimately pushes costs up and quality down compared to data provided by the originating CNA or vendor.
• Importance of Transparency and "Showing Your Work": A core tenet of Vulnrichment is transparency, with all processes, documentation, and issue resolution occurring publicly. This approach not only fosters trust and community input but also makes it easier to defend analytical conclusions. The ability to publicly test hypotheses, such as the correlation between high CVSS scores and KEV inclusion, provides valuable data for internal CISA decision-making.

Technical Deep Dive

▶ Watch: Overview of CISA's SSVC, KEV, and CSAF frameworks (5:40)

The Vulnrichment initiative is built on a structured process for augmenting CVE records with additional, valuable security information. Its technical architecture and operational workflow are designed to integrate CISA's internal analysis with the broader CVE ecosystem while upholding principles of transparency and efficiency.

Data Sources and Enrichment Logic:

Vulnrichment's primary function is to provide specific types of data, prioritizing its internal analysis and filling gaps where originating CNAs have not provided information:

• SSVC and KEV Information: These are CISA-originated data points. The SSVC framework is used internally by CISA to score and prioritize vulnerabilities for federal agencies. If a vulnerability is added to the KEV Catalog, Vulnrichment ensures this flag is inherited into the enriched CVE record. This information is directly published by CISA through Vulnrichment.
• CVSS and CWE: Vulnrichment provides CVSS scores and CWE identifiers only if the originating CNA has not already supplied them. A key policy dictates that if a CNA later provides this information, Vulnrichment's data is rescinded in favor of the CNA's authoritative assessment. This avoids conflicts and upholds the CNA as the primary source of truth for their vulnerabilities.
• Additional References: Vulnrichment occasionally adds relevant URLs as additional references to CVE records. The criteria for inclusion are typically that the URL is obvious, easily accessible, and adds clear value to understanding the vulnerability.
• CPE (Common Platform Enumeration): This was a significant area of experimentation. From April to December 2024, Vulnrichment attempted to provide CPE information. The process involved a hierarchy: first checking the NVD's dictionary of CPEs, then looking for CPEs within the broader NVD corpus that might not be in the official dictionary, and finally, as a last resort, attempting to "make one up." This experiment revealed profound difficulties in ensuring semantic correctness and consistency with the NVD's authoritative dictionary, leading to its discontinuation by Vulnrichment. Currently, CPE is provided directly by CNAs (though few do) or by the NVD. It's noted that the CVE record format has evolved to better support full CPE information, including ranges and versions, which was previously "vestigial."

Operational Process:

The Vulnrichment process follows a clear, sequential flow:

1. Triage and SSVC: Newly published CVEs are ingested and undergo CISA's internal SSVC scoring process.
2. KEV Flagging: If a CVE is identified as being in the KEV Catalog, it is flagged accordingly.
3. Data Gap Analysis: The system checks for missing CVSS, CWE, or critical references.
4. Enrichment: If data gaps are found, Vulnrichment adds the relevant information based on CISA's analysis.
5. Publication: The enriched data is published in two ways:

• An ADP (Authorized Data Publisher) container from CISA Vulnrichment is added to and updated within the official CVE corpus. This ensures the enriched data becomes part of the authoritative CVE record.
• The Vulnrichment information is also made available in a separate GitHub repository, distinct from the main CVE repository.

Transparency and Community Interaction:

Transparency is a cornerstone of Vulnrichment's technical implementation:

• Public GitHub Repository: All enrichment data, documentation, and the process itself are publicly accessible on GitHub. This repository serves as the primary interface for community interaction.
• Issue Resolution: Community members can submit Pull Requests (PRs) and issues to suggest corrections or improvements. The team actively monitors and responds to these, with an average resolution time of 6.5 days and an average of 2.1 comments per issue/PR. While PRs don't directly modify the upstream CVE data, the team ensures proposed changes are synced on the backend.
• "Stop Arguing" Policy: A deliberate policy exists to avoid prolonged debates on CVSS, CWE, or CPE scores. If a CNA has provided a score, Vulnrichment accepts it. The focus is on filling missing information rather than auditing or disputing existing authoritative data. This pragmatic approach aims to prevent resource drain and maintain focus on the core mission of widespread enrichment.

The technical deep dive highlights Vulnrichment's role as an agile, responsive mechanism for improving the quality and completeness of vulnerability data, demonstrating CISA's commitment to collaborative and transparent security practices.

Demo / Proof of Concept

▶ Watch: Vulnrichment's goal: democratizing vulnerability analysis (6:40)

The talk "Vulnrichment: Year One" did not feature a live technical demonstration or proof of concept in the traditional sense. Instead, the speakers focused on presenting the operational data, processes, and lessons learned from the first year of the Vulnrichment initiative. They utilized charts to illustrate trends in CVE publication and enrichment rates, and screenshots of GitHub issues and discussions to exemplify community engagement and the team's responsiveness. The narrative itself served as an extensive overview of the program's practical application and its impact on the vulnerability ecosystem.

Defensive Implications

▶ Watch: Initial design goals for the Vulnrichment project (7:40)

The Vulnrichment initiative provides several critical implications and actionable insights for defenders seeking to enhance their vulnerability management strategies:

• Prioritize with Enriched Data: Defenders should actively leverage the enriched data provided by Vulnrichment. The inclusion of SSVC scores offers CISA's prioritization perspective, which is invaluable for understanding the criticality of vulnerabilities, particularly for federal and critical infrastructure entities. Similarly, the KEV Catalog flag is a direct signal for vulnerabilities that are actively being exploited and require immediate attention, often mandated by BOD 22-01.
• Fill Information Gaps: When evaluating CVEs, defenders should cross-reference information with Vulnrichment data, especially for CVEs where the originating CNA has not provided CVSS scores, CWE identifiers, or sufficient references. This additional context can significantly improve risk assessment and inform patching decisions.
• Engage with Transparency: The public GitHub repository for Vulnrichment offers an unprecedented level of transparency. Defenders can monitor discussions, review change requests, and even submit issues if they identify discrepancies or have additional information for a specific CVE. This direct engagement fosters a more collaborative and informed defense community.
• Understand Data Source Authority: It's crucial for defenders to understand Vulnrichment's policy: it fills gaps but does not override CNA-provided data. This means that if a CNA has provided a CVSS score, Vulnrichment will not dispute it. Defenders should always prioritize the authoritative source (the CNA/vendor) where available, using Vulnrichment as an enhancement.
• Advocate for Upstream Enrichment: The finding that non-authoritative enrichment increases cost and lowers quality underscores the importance of encouraging vendors and CNAs to provide comprehensive vulnerability data at the "birth" of a CVE. Defenders can advocate for this practice within their supply chains and with their software providers.
• Automate with Machine-Readable Formats: The availability of CISA advisories in CSAF format, combined with the structured data from Vulnrichment, enables greater automation in vulnerability management. Defenders should integrate these machine-readable feeds into their security tools and workflows to streamline vulnerability identification, assessment, and response.
• Acknowledge the Software Identification Challenge: The experience with CPE highlights the persistent problem of "globally naming software." Defenders should be aware of the inherent complexities and potential inconsistencies in software identification across different data sources and build their asset management and vulnerability scanning strategies with this challenge in mind, potentially relying on multiple identification methods.

By actively integrating Vulnrichment's contributions and understanding its operational principles, defenders can build more robust, informed, and responsive vulnerability management programs.

Key Takeaways

• Democratization of CISA's Analysis: Vulnrichment serves as a crucial public service, democratizing CISA's internal vulnerability analysis, including SSVC prioritization and KEV Catalog inclusion, to enrich CVE records with valuable context.
• Transparency and Community Engagement: A public GitHub repository is central to Vulnrichment's operations, fostering transparency, enabling community feedback, and facilitating rapid resolution of data issues, exemplified by swift responses to concerns raised by the community.
• Pragmatic Enrichment, Not Auditing: CISA's Vulnrichment initiative focuses on filling critical data gaps (like missing CVSS and CWE) in CVE records, rather than auditing or overriding existing information provided by originating CNAs, adhering to a policy of not engaging in protracted scoring debates.
• Value of Experimentation: The program embraces an experimentation mindset, as demonstrated by the CPE enrichment trial. Even when experiments highlight unsolved ecosystem challenges (like the difficulty of "globally naming software"), they yield valuable operational lessons.
• Push for Upstream Data Provision: A long-term goal for the vulnerability ecosystem is for CNAs and software maintainers to provide comprehensive enrichment data directly at the time of CVE record creation, as non-authoritative enrichment is less efficient and potentially less accurate.
• Actionable Intelligence for Defenders: Defenders should actively integrate Vulnrichment's enriched CVE data—including SSVC, KEV status, and added CVSS/CWE—into their vulnerability management processes to enhance prioritization, improve risk assessments, and drive more effective security responses.

About the Speaker(s)

The talk "Vulnrichment: Year One" was presented by Art and Lindsay, both key contributors to the U.S. Cybersecurity and Infrastructure Security Agency (CISA). They are integral members of CISA's vulnerability management subdivision within the cybersecurity division, focusing on reducing the prevalence and impact of vulnerabilities across federal, state, local, tribal, territorial governments, and critical infrastructure.

Lindsay, who has been with CISA for nearly five years, highlighted her background, including a former life as a high school teacher, which she humorously referenced when discussing the importance of "showing your work" in vulnerability analysis. She plays a significant role in CISA's operational lead for vulnerability response, including the agency's long-standing Coordinated Vulnerability Disclosure (CVD) efforts and the implementation of frameworks like SSVC and the KEV Catalog.

Art is also deeply involved in the Vulnrichment team, contributing to the technical implementation and strategic direction of the program. His remarks often focused on the practical challenges and policies of data enrichment, such as the complexities of CPE and the team's approach to CVSS scoring debates. Both speakers demonstrated a profound understanding of the vulnerability ecosystem and CISA's commitment to transparency and continuous improvement in public service.