Boosting Vulnerability Intelligence: How Accurate CWE Mappings Transform ML Model Performance

Boosting Vulnerability Intelligence: How Accurate CWE Mappings Transform ML Model Performance

Speakers: David Starobinski (Boston University); Sevval Simsek (Boston University); Varsha Athreya (Boston University)

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=vV3kftnWkfg

Overview

In the realm of cybersecurity, the accuracy and depth of vulnerability intelligence are paramount for effective defense. This talk, presented by David Starobinski, Sevval Simsek, and Varsha Athreya from Boston University at VulnCon, addresses a critical flaw in current vulnerability data: the prevalence of inaccurate or generic CWE (Common Weakness Enumeration) mappings within the NVD (National Vulnerability Database). The speakers highlight that a significant portion of CVEs are mapped to "discouraged" or "prohibited" CWEs, which provide little actionable insight into the true root cause of vulnerabilities. This widespread data quality issue not only hinders human understanding but also severely compromises the performance of Machine Learning (ML) models trained on this intelligence.

The core of their presentation introduces Fix V2W, an innovative algorithm designed to automatically generate more precise CWE mappings. Leveraging knowledge graphs and knowledge graph embeddings, Fix V2W aims to transform the quality of vulnerability intelligence. By providing more granular and accurate root cause information, the researchers demonstrate how their approach can improve ML model performance, potentially prevent exploitation by identifying true weaknesses earlier, and reduce the manual effort involved in vulnerability remediation. This work underscores the vital link between clean, structured data and robust security operations in an increasingly AI-driven landscape.

Background

▶ Watch: Introduction: The problem of CWE mapping (0:00)

The National Vulnerability Database (NVD) serves as a cornerstone for vulnerability intelligence, enriching CVE (Common Vulnerabilities and Exposures) entries with crucial metadata such as CWEs (Common Weakness Enumerations), CPEs (Common Platform Enumerations), and references. This enrichment is a community effort, with CNAs (CVE Numbering Authorities) initially providing information, and the NVD stepping in to fill gaps. However, a significant problem persists in the quality of CWE mappings, which are intended to identify the root cause of vulnerabilities.

The speakers categorize these problematic mappings into several types. Discouraged CWEs, while technically valid, are overly generic and high-level, failing to provide specific root cause information. An example cited is CWE-20: Improper Input Validation, a common but broad classification. Despite being "discouraged," CNAs frequently use them, leading to a continuous increase in such mappings as the number of CVEs grows annually. Even worse are Prohibited CWEs, which are not actual weaknesses but rather collections or categories of other CWEs (e.g., "Numeric Errors"). These offer no concrete root cause. Fortunately, new CVEs have not been mapped to prohibited CWEs since 2019, but existing mappings remain within the NVD. Additionally, many CVEs are mapped to placeholder CWEs or remain entirely unmapped.

The scale of this issue is striking: as of early 2024, out of approximately 280,000 CVEs in the NVD, less than half possessed valid and specific CWE mappings. The remaining majority were linked to placeholder, prohibited, or discouraged categories. This data quality problem has tangible implications. The researchers found that CVEs with these invalid mappings are not merely academic concerns; they are actively exploited and appear in CISA's Known Exploited Vulnerabilities (KEV) catalog. One example mentioned was a CVE exploited shortly after its addition to the database, suggesting that an inadequate understanding of its root cause led to improper remediation. Furthermore, only 37% of exploited CVEs had valid CWE mappings, highlighting a critical gap in actionable intelligence for the most dangerous vulnerabilities.

This pervasive lack of precise root cause information directly impacts the effectiveness of security initiatives. When ML models are trained on such flawed data, their ability to accurately predict, prioritize, or even understand vulnerabilities is severely hampered. Without spending "days or weeks picking through this data," security practitioners and automated systems are forced to operate with a compromised understanding of the threats they face. The existing CWE hierarchy, with its parent-child relationships, provides a structural foundation, but the problem lies in the inconsistent and often generic application of these classifications. This forms the crucial context for the proposed solution: a method to systematically rectify these mappings and boost the overall quality of vulnerability intelligence.

Key Findings

▶ Watch: Proposed solution: Knowledge graphs for better mappings (6:00)

The research presented by the Boston University team reveals several critical findings concerning the state of vulnerability intelligence and the potential for its improvement:

• Pervasive Invalid Mappings: A substantial majority of CVE (Common Vulnerabilities and Exposures) entries in the NVD (National Vulnerability Database) suffer from invalid or overly generic CWE (Common Weakness Enumeration) mappings. As of early 2024, less than half of the 280,000 CVEs had valid, precise CWEs, with the rest mapped to placeholder, prohibited, or discouraged categories. This significantly degrades the quality of vulnerability intelligence.

• Impact on Exploited Vulnerabilities: The problem extends to high-severity, actively exploited vulnerabilities. The study found that only 37% of CVEs listed in CISA's Known Exploited Vulnerabilities (KEV) catalog had valid CWE mappings, indicating that even critical vulnerabilities often lack proper root cause identification. This suggests that inadequate root cause understanding may contribute to ineffective remediation and continued exploitation.

• Fix V2W Algorithm Efficacy: The developed Fix V2W algorithm, which utilizes knowledge graphs and knowledge graph embeddings, demonstrates significant success in predicting accurate CWE mappings. When evaluated against historical NVD remap data (2021-2024), the algorithm correctly predicted the remapped CWE within its top 10 ranked suggestions for 60-70% of the cases.

• Proactive Exploitation Avoidance: A particularly impactful finding is the algorithm's ability to proactively identify correct mappings for exploited CVEs. For 69% of CVEs that were remapped after being exploited, Fix V2W could have provided the correct mapping before the exploitation occurred. This highlights the potential to shift from reactive to proactive vulnerability management.

• Improved ML Model Performance: Integrating the improved CWE mappings generated by Fix V2W into the knowledge graph significantly boosts the performance of downstream ML models. The researchers observed a "large increase in model performance," particularly when using the top two ranked CWE candidates from their algorithm, measured by the Hits@K metric. This confirms that better quality input data directly translates to more effective ML-driven security tools.

• CWE Relationship Closeness: An analysis of NVD remappings between 2016 and 2024 revealed that old and new CWE mappings for the same CVE are often closely related within the CWE hierarchy. Approximately 60% of remapped CWEs were found to be within two "hops" (e.g., child, parent's child, or sibling) of the original, invalid mapping. This insight informs the design of the candidate selection process in Fix V2W.

• Mitigation of Node Centrality Bias: The algorithm effectively addresses the "node centrality" problem prevalent in highly popular CWEs (like those in the Top 25 list). By strategically including or excluding these popular CWEs from the candidate list based on their relevance to the specific vulnerability, Fix V2W avoids skewing predictions towards overly generalized categories.

Technical Deep Dive

▶ Watch: Impact: Predicting correct CWEs before exploitation (15:00)

The core of the proposed solution, Fix V2W, is a sophisticated, data-driven approach designed to enhance the accuracy of CWE (Common Weakness Enumeration) mappings in the National Vulnerability Database (NVD). This method is built upon three main pillars: leveraging existing knowledge, constructing a knowledge graph, and applying knowledge graph embeddings to learn relationships, finally culminating in a refined ranking algorithm.

Knowledge Graph Construction

The first step involves building a comprehensive knowledge graph that captures the intricate relationships within vulnerability data. This graph is not merely a collection of isolated facts but a structured representation where entities are nodes and their relationships are edges. The primary entities (nodes) in this graph include:

• CWEs: Representing the root causes of vulnerabilities.
• CVEs: Specific vulnerabilities identified.
• Products: The software or hardware affected by a CVE.

Relationships (edges) within the graph are crucial for inferring connections and context:

• Parent-of: Reflecting the hierarchical structure of CWEs, where more generic CWEs are parents to more specific ones. This allows the model to understand the conceptual breadth and depth of weaknesses.
• Member-of: Used for prohibited CWEs, which are categories rather than specific weaknesses, linking them to their constituent valid CWEs.
• Affected-by: Connecting CVEs to their mapped CWEs.
• Found-in: Linking CVEs to the products they impact.

By establishing these connections, the knowledge graph ensures that entities sharing common attributes or relationships (e.g., two CVEs mapped to the same CWE) are conceptually "close" within the graph structure, even if they appear disparate in raw data.

Knowledge Graph Embeddings

Once the knowledge graph is constructed, the next phase is to use knowledge graph embeddings to learn the latent relationships within this structure. This technique maps each entity (CWE, CVE, Product) and relationship type into a continuous, multi-dimensional vector space (e.g., over 100 dimensions, though often visualized in 2D for simplicity). The goal is to represent entities such that their spatial relationships in the vector space reflect their semantic and structural relationships in the graph.

The model is trained to minimize a loss score. It "rewards" itself for correctly predicting known facts (triples like (CVE, affected-by, CWE)) and "penalizes" itself for incorrectly predicting false facts. Through this iterative optimization process, the model learns a dense representation where valid triples have low "distance" or high "score" in the embedding space, while invalid triples have high distance or low score.

The researchers experimented with several embedding models, ultimately finding the TransE model to be most effective for their data. A key advantage of their approach, particularly with TransE, is that it relies solely on the relationships between CVE IDs, CWE IDs, and CPE IDs, without incorporating semantic definitions or descriptive text. This makes the model "lightweight" and efficient, avoiding the complexities of natural language processing (though they acknowledge LLMs could be explored for hard-to-map cases in the future). Previous attempts to include additional data like CVSS scores actually diminished performance, reinforcing the focus on relational data.

Fix V2W Algorithm for Candidate Ranking

The knowledge graph embeddings provide a powerful mechanism for scoring potential relationships, but generating a relevant list of candidate CWEs is equally important. The researchers identified issues with traditional approaches:

• Using the entire CWE list (CWE 1000 tree): Too many candidates, leading to computational overhead and potential noise.
• Using the Top 25 CWE list: While popular, these highly central nodes can skew predictions due to the node centrality problem, where the graph favors connections to already well-connected nodes, even if irrelevant to a specific CVE.

To address this, the Fix V2W algorithm dynamically determines a specialized set of "allowed CWEs" for each CVE being remapped. This process involves:

1. Identifying the "old" (invalid) CWE mapping: For a CVE mapped to a prohibited or discouraged CWE (e.g., "Numeric Errors").
2. Extracting category members: If the old CWE is a prohibited category, the algorithm identifies its valid member CWEs (e.g., Integer Overflow, Buffer Overflow). The system prioritizes members that NVD currently uses for mapping.
3. Including children CWEs: If these member CWEs (or the initial discouraged CWE) have more specific "children" in the CWE hierarchy, these children are also added to the candidate set. The rationale is that if a broader category is relevant, its more specific descendants are likely even better candidates.
4. Filtering for relevance: To mitigate the node centrality problem, highly popular CWEs (like those in the Top 25) are only included in the candidate list if they are genuinely related to the "old" CWE mapping of the vulnerability in question. This prevents irrelevant popular CWEs from dominating the predictions.

Once this tailored candidate list is generated, the knowledge graph embedding model calculates a score for each candidate CWE, representing the likelihood of it being the correct root cause for the given CVE. These candidates are then rank-ordered by their scores, providing a top N list (e.g., top 5 or top 10) of most likely CWEs. The researchers acknowledge that while the top-ranked CWE is often correct, human analysts may still need to review the list, as the second or third suggestion could be more appropriate, or even multiple CWEs could apply. This blend of automated prediction and informed manual review represents a practical approach to boosting mapping accuracy.

Evaluation Methodology

To validate their model, the team employed a rigorous evaluation strategy:

• Data Split: All CVEs from 1999 to 2024 were divided. The model was trained on CVEs mapped between 1999 and 2021.
• Test Set/Ground Truth: The evaluation focused on remaps that occurred between 2021 and 2024. These NVD-initiated remaps served as the ground truth for assessing the algorithm's ability to predict correct, updated mappings.
• Metric: The primary metric used was Hits@K, which measures the percentage of new mappings that are correctly predicted within the top K ranks of the algorithm's suggestions. This metric effectively captures the model's ability to present the correct answer among its top choices.

The results demonstrated that the model could successfully predict the correct remapping within the top 10 predictions for 60-70% of the cases. Interestingly, they noted instances where their model's prediction, though counted as "wrong" against the NVD's remapping, appeared to be a more accurate and specific root cause. For example, a privilege escalation CVE remapped by NVD to "Incorrect Comparison" was predicted by their model as "Privilege Dropping and Lowering Errors," which could be a more precise description. The observation that using the "top two" candidate CWEs yielded better performance than just the "top one" also highlighted the nuance of the ranking, often due to the persistent bias of very popular CWEs in the first position.

Demo / Proof of Concept

▶ Watch: Automated pipeline for continuous model updates (17:00)

To demonstrate the practical applicability and ensure the continuous relevance of their Fix V2W algorithm, the research team developed an automated pipeline workflow. This pipeline addresses the dynamic nature of the NVD (National Vulnerability Database), which is constantly updated with new CVEs and remaps, necessitating continuous model retraining.

The entire pipeline is built and deployed on Red Hat OpenShift AI, leveraging its robust features for machine learning operations. Key components of this platform utilized include:

• GPUs: Essential for the computationally intensive training phase of the knowledge graph embedding model.
• Model Server: For deploying the trained model, making it accessible via an external API endpoint for inference requests.
• Object Storage: For storing the knowledge graph data, embeddings, and other necessary metadata.

The workflow operates on a continuous integration/continuous delivery (CI/CD) paradigm, allowing for regular updates:

1. Local NVD Version: The pipeline starts by identifying the current local version of the NVD data that the model was last trained on.
2. Scraping NVD Change History: It then queries the NVD change history API to fetch all updates (new CVEs, remaps, etc.) that have occurred since the local version's date up to the current date.
3. Applying Changes: These newly collected updates are applied to the local NVD dataset, ensuring the knowledge graph reflects the latest vulnerability information.
4. Model Retraining: The knowledge graph embedding model is then retrained on this updated dataset. This step leverages the GPUs provided by OpenShift AI for efficiency.
5. Model Deployment: After successful retraining, the updated model is deployed to the OpenShift AI model server, and the associated data is updated in object storage.
6. Iteration: This entire process can be scheduled to run automatically on a regular basis, such as weekly or monthly, ensuring the model's predictions remain current and accurate.

During the presentation, a simple front-end interface was demonstrated to illustrate the pipeline's functionality. This interface allows users to:

• Query for CVE-CWE mappings: Specifically targeting discouraged or prohibited mappings within a stored tabular database.
• Select a CVE: An example was shown where a CVE currently mapped to CWE-400 (a prohibited or discouraged CWE) was selected.
• Run Inference: The system then sends a request to the model server and utilizes the cluster storage for metadata, running the Fix V2W algorithm.
• Output Top Predictions: The front-end displays the top 10 predicted CWEs, ranked by likelihood, providing potential accurate root cause mappings.

The benefits of this automated pipeline are significant:

• External API Endpoint: Provides easy programmatic access to the Fix V2W inference capabilities for integration into other security tools.
• Continuous Automated Retraining: Eliminates the manual effort and ensures the model is always up-to-date with the latest NVD information.
• Reduced Manual Remap Time: Significantly streamlines the process of identifying and correcting invalid CWE mappings.
• Lightweight Inference: While GPUs are crucial for training, inference (making predictions) is fast and does not require a GPU, making it efficient for real-time lookups.

Crucially, the researchers emphasized that their work is open source, with a tutorial and code available on GitHub. This open approach encourages collaboration and allows other organizations and researchers to adopt and integrate Fix V2W into their own vulnerability management workflows, further validating its utility and impact.

Defensive Implications

▶ Watch: Q&A: Details on the knowledge graph embedding model (25:30)

The findings and proposed solution from the Boston University team have profound implications for defensive cybersecurity strategies, offering practical ways to bolster an organization's resilience against vulnerabilities.

Firstly, the primary benefit lies in improved vulnerability intelligence. By transforming generic or invalid CWE (Common Weakness Enumeration) mappings into precise root causes, security teams gain a much clearer understanding of the underlying weaknesses in their software and systems. This granular insight is critical for effective remediation. Instead of patching symptoms or addressing broad categories like "Improper Input Validation" (CWE-20), defenders can target specific issues such as "Integer Overflow" or "Buffer Overflow," leading to more robust and permanent fixes. This accuracy directly translates to better prioritization of vulnerabilities, ensuring that critical resources are focused on the most impactful weaknesses.

Secondly, the algorithm's demonstrated ability to proactively avoid exploitation is a game-changer. The study showed that for 69% of CVEs that were eventually exploited, Fix V2W could have identified the correct, specific CWE mapping before the exploitation occurred. This means organizations could potentially identify the true nature of a vulnerability much earlier in its lifecycle, enabling them to implement targeted defenses or patches before attackers can leverage it. This shift from reactive firefighting to proactive defense is invaluable in minimizing an organization's attack surface and reducing the window of vulnerability.

Thirdly, the improvement in data quality directly enhances ML-driven security tools. Many modern security solutions, including vulnerability scanners, threat intelligence platforms, and risk prioritization engines, rely heavily on the NVD (National Vulnerability Database) for context and training data. When these tools are fed with cleaner, more accurate CWE mappings, their performance, predictive capabilities, and overall reliability improve significantly. This leads to more accurate risk assessments, better automated responses, and more intelligent security operations, moving closer to the promise of AI in cybersecurity.

Fourthly, the automated pipeline for continuous retraining and inference offers significant resource optimization. The manual effort required to correctly map and re-map vulnerabilities is substantial. By automating this process, security analysts can be freed from tedious data curation tasks, allowing them to focus on higher-value activities such such as threat hunting, incident response, and strategic security architecture. This efficiency gain is particularly important in an industry often plagued by a shortage of skilled personnel.

Finally, the enhanced specificity of CWE data supports more effective strategic patching and secure coding initiatives. With precise root cause information, development teams can improve their secure coding practices, implement more targeted security controls, and design more resilient software from the outset. This moves beyond simply fixing individual vulnerabilities to addressing systemic weaknesses, fostering a culture of security by design. Furthermore, better CWE data can inform threat modeling exercises, allowing organizations to identify common weakness patterns across their applications and infrastructure, and build defenses accordingly.

Key Takeaways

• The National Vulnerability Database (NVD) suffers from widespread inaccuracies in CWE (Common Weakness Enumeration) mappings, with less than half of CVEs having valid, specific root cause classifications.
• This prevalence of "discouraged" and "prohibited" CWEs hinders effective vulnerability intelligence, impacts actively exploited vulnerabilities, and degrades the performance of Machine Learning (ML) models trained on this data.
• The Fix V2W algorithm, leveraging knowledge graphs and TransE knowledge graph embeddings, significantly improves the accuracy of CWE root cause mapping, achieving 60-70% accuracy in predicting remapped CWEs within the top 10 suggestions.
• Accurate CWE mappings generated by Fix V2W can proactively identify and address vulnerabilities, with the potential to predict correct mappings for 69% of exploited CVEs before their exploitation.
• Integrating Fix V2W's top two predicted CWE candidates into the knowledge graph demonstrably boosts the performance of ML models, highlighting the critical link between data quality and model efficacy.
• An automated, continuous pipeline built on Red Hat OpenShift AI ensures the mapping model stays updated with the dynamic NVD, offering an external API endpoint, reducing manual effort, and enabling lightweight inference for practical deployment.

About the Speaker(s)

The research presented was a collaborative effort from Boston University, with contributions from David Starobinski, Sevval Simsek, and Varsha Athreya. David Starobinski is likely a faculty member or senior researcher, guiding the academic direction of the work. Sevval Simsek and Varsha Athreya are presented as talented PhD students from Boston University, who are actively involved in this research and are nearing graduation, currently undergoing extensive interviews in the field.

Their work is supported and primarily funded by the Red Hat Collaboratory at Boston University and the US National Science Foundation, highlighting a strong partnership between academia and industry. The team also acknowledged other contributors including Hu Alger, Jonathan Luke, and David Sastre Medina from Red Hat, who served as a mentor. The researchers are actively involved with the CWE/Mitre root cause mapping working group, indicating their commitment to impacting industry standards. They emphasized their open-source approach, encouraging collaboration and adoption of their tools, rather than selling a commercial product.