Who’s Vulnerability Is It Anyway? Harmonizing Stakeholder Roles in Vulnerability Management

Who’s Vulnerability Is It Anyway? Harmonizing Stakeholder Roles in Vulnerability Management

Speakers: Yam Yam (Zscaler), Kayla Under Coffler (Zenedity), Hava (Sneak), James (Laciotech)

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=1d3hG1-SINI

Overview

This panel discussion, born from spontaneous conversations at VulnCon 2024, delves into the intricate and often contentious landscape of vulnerability management. Moderated by Yam Yam, the panel brings together diverse perspectives from a threat research lead, a vulnerability management operator, a security analyst (also representing developers), and a security architect/executive. The core focus is to dissect the complexities, challenges, and inherent friction points arising from the varied priorities and operational realities of different stakeholders involved in the vulnerability lifecycle.

The talk highlights a critical disconnect within organizations: while security teams are often measured on the sheer volume of vulnerabilities identified and remediated, developers prioritize rapid feature delivery and business deadlines. This fundamental misalignment, coupled with issues like false positives, overlooked false negatives, and ambiguous ownership for remediation, creates significant operational inefficiencies and can undermine an organization's actual security posture. The panel aims to shed light on these tensions and propose strategies for fostering better collaboration and ultimately, more effective risk reduction.

Ultimately, this discussion is crucial for any organization grappling with the relentless tide of new vulnerabilities. It emphasizes that effective vulnerability management transcends mere scanning and reporting; it requires a holistic approach that integrates technical understanding, business context, clear communication, and a shared sense of ownership across all teams to move beyond a reactive "hamster wheel" and towards proactive, risk-informed decision-making.

Background

▶ Watch: Panel introduction: Harmonizing stakeholder roles in vulnerability management (0:00)

The genesis of this panel discussion emerged from candid conversations between Yam Yam and Kayla Under Coffler at VulnCon 2024, where they recognized a shared frustration regarding the fragmented nature of vulnerability management. They observed that while many practitioners understood their specific piece of the puzzle, a comprehensive, unified understanding of the entire process, encompassing all personas, was often lacking. This insight led to the formation of a panel designed to bring these disparate voices together.

The problem at its heart is the inherent diversity of roles involved in vulnerability management, each with distinct priorities, skill sets, and metrics of success.

• The Vulnerability Management Operator (represented by Kayla) is typically focused on ensuring scanners run accurately, cover all assets, and communicate findings to the right stakeholders. Their success is often measured by scan completion rates and the flow of vulnerability data, rather than the ultimate remediation outcome. Kayla vividly described this as "cat herding" when dealing with asset inventory and asset management, often not being directly responsible for these foundational elements, yet their job heavily depended on them.
• The Developer (represented by Hava's dual perspective) primarily aims to "move fast, keep up with deadlines, and work according to planning requirements." For them, fixing vulnerabilities can feel like an "additional task" or a "waste of time," especially if the reported issues lack context or seem irrelevant to their specific application. Their key questions revolve around the actual relevance and impact of a vulnerability: "Is this even a real vulnerability or is this relevant to me?"
• The Security Analyst (represented by Hava) acts as a crucial bridge, striving to provide developers with "actionable" and "complete" information, focusing on details like fixed versions, commit IDs, and vulnerable conditions that make a vulnerability exploitable in a specific context. Their goal is noise reduction and ensuring accuracy and timeliness in disclosures.
• The Executive/Security Architect (represented by James) operates at a higher level, balancing compliance and technical requirements with broader business continuity. This role involves translating complex security risks into business language, navigating the friction between development teams (who report false positives and delays) and security teams (who highlight critical vulnerabilities and high risk). Executives are often caught between the desire to adopt leading-edge security tools and the practical reality of managing the downstream impact on development velocity and organizational goals. James highlighted the challenge of communicating high-level business risk without getting immediately mired in technical specifics like ML BOMs, CVE counts, and transitive dependencies.

This backdrop sets the stage for a discussion on how these differing priorities and metrics create friction, lead to misunderstandings, and ultimately hinder effective vulnerability management, preventing organizations from truly reducing their security risk.

Key Findings

▶ Watch: Kayla introduces the Vulnerability Management Operator persona (2:30)

The panel discussion illuminated several critical findings regarding the state of vulnerability management, underscoring the complexities that arise from diverse stakeholder perspectives:

1. Divergent Priorities and KPIs Drive Friction: The most prominent finding was the fundamental misalignment of priorities and Key Performance Indicators (KPIs) across different roles. Kayla, as the VM operator, was measured on scan accuracy and communication, not necessarily remediation. Developers, as Hava explained, are measured on meeting deadlines and delivering features, with vulnerabilities often seen as disruptive backlog items. James highlighted that executives struggle to reconcile security's focus on vulnerability counts with business drivers like product adoption and revenue. This disconnect creates a "hamster wheel" effect, where identifying vulnerabilities becomes an endless task without sufficient focus on actual risk reduction.

1. The "False Positive" Dilemma is Contextual: False positives are a perennial problem, but the panel revealed that their definition and impact vary significantly. Kayla initially adopted a strict "trust your scanner" approach, demanding developers prove a false positive. Hava, however, defined a false positive as a real vulnerability that is "not relevant in the context of your application," emphasizing the need for runtime factors and environment visibility. James added that developers often view a vulnerability as a false positive if it's not immediately exploitable, even if a vulnerable version is present. This highlights that the issue isn't just scanner accuracy but the lack of contextual understanding and a shared definition of what constitutes a "real" problem.

1. False Negatives are Security Blind Spots: While false positives are annoying, the panel argued that false negatives are far more dangerous as they create critical "security blind spots." Yam shared research indicating that many scanners miss significant vulnerabilities, emphasizing that "everybody lies" in terms of comprehensive detection. Hava stressed that "when you don't know there's a vulnerability, then you don't get the chance to decide on what to do next." James extended this to supply chain security, noting that major incidents like XZ Utils or TJ actions often bypass traditional CVE scanning entirely, requiring a deeper look at open-source usage patterns.

1. Remediation is Underprioritized and Inefficient: Remediation, the ultimate goal of vulnerability management, is often the weakest link. Kayla, as an operator, was never the remediator, relying on teamwork to figure out fixes. James argued that remediation guidance is severely underprioritized. Security teams often think in terms of individual CVEs and CVSS scores, while developers think in terms of package upgrades and architectural changes. This leads to inefficient, duplicated efforts (e.g., 30 teams independently patching a Spring framework CVE) instead of coordinated, package-level migrations. The discussion also touched on the complex "ownership" debate, where maintainers might argue a fix is a developer's responsibility (e.g., input sanitization), further complicating remediation.

1. Risk Reduction Trumps Vulnerability Count: The panel strongly advocated for shifting the focus from vulnerability reduction (a sheer count) to risk reduction. Hava pointed out that while CVSS provides a common language for severity, it's often insufficient in a context-driven era. She introduced EPSS (Exploit Prediction Scoring System) as a valuable addition for understanding the likelihood of exploitation, alongside business impact and reachability. James reinforced this by stating that CVEs can be a "crutch" for a true understanding of software deployments and underlying infrastructure (e.g., Kubernetes). Kayla added that an "over-rotation on reporting" vulnerability numbers can become a "cultural issue," obscuring actual risk.

1. Tooling Challenges and Vendor Incentives: A recurring theme was the love-hate relationship with security tools. James critiqued the sheer number of tools, their difficulty in setup and management, and their primary focus on identification rather than actual risk reduction. He also pointed out that vendors are often incentivized to show high detection numbers (which security teams ask for), even if it leads to an abundance of false positives. Kayla expressed a common desire for "one scanner to rule them all," acknowledging its unlikelihood but highlighting the complexity of aggregating data from disparate tools speaking "different languages" at various stages of the SDLC.

Technical Deep Dive

▶ Watch: Hava details the developer's view on fixing vulnerabilities (3:40)

The technical discussions revolved around the practicalities of vulnerability discovery, assessment, and remediation, highlighting the gaps and opportunities for more effective practices.

Vulnerability Scanning and Aggregation

The panel acknowledged the necessity of multiple scanning tools, but also the inherent challenges. Kayla articulated the common desire for "one scanner to rule them all," lamenting that different tools "speak different languages" and identify vulnerabilities at various stages of the Software Development Life Cycle (SDLC). This necessitates building custom aggregation capabilities to consolidate CVE data with asset information, ensuring findings reach the correct owners in an understandable format.

James offered a nuanced approach to scanning, advocating for a separation between the developer experience and the security engineer's deeper analysis. For developers, he preferred lightweight hooks integrated into their environment, such as SCA (Software Composition Analysis) tools scanning GitHub repos. While acknowledging these might "miss some things," the goal is to foster a mindset of patch management and prudent package usage early in the development cycle. For the security team, James recommended deeper analysis scanning, including direct binary analysis and runtime function-level reachability. This comprehensive visibility, though more complex to unravel, provides the "source of truth" for understanding where vulnerabilities truly exist and are exploited within the environment, especially crucial during critical events like Log4j. He emphasized the need for visibility at "every step of the way" within the CI/CD pipeline, from binary build to deployment, to account for potential injections or modifications. The ultimate aggregation point, he noted, is not just collecting data but effectively prioritizing findings and delivering "relevant finding to the relevant developer" without causing undue friction or confusion, especially for junior developers who may not fully grasp the deployed architecture.

Context-Driven Vulnerability Assessment

A significant technical theme was the move beyond raw vulnerability metadata to a more context-driven assessment. Hava stressed that while scanners identify vulnerabilities, understanding if an issue is "an actual vulnerability in the context of the application" is paramount. She cited examples where vulnerabilities are relevant "only on a specific OS or on a specific environment," and without that context, developers waste time fixing irrelevant issues. This requires visibility into runtime factors and the specific environment where the vulnerability exists.

James further elaborated on the developer's perspective, noting they often think in terms of exploitability rather than simply the presence of a vulnerable version. He argued that the industry's focus on reachability and similar tools is an attempt to answer this, but at scale, it's impractical to create a "satisfactory paper trail for every one of these vulnerabilities to prove that it's an exploitable issue or not." Instead, he suggested the broader goal should be robust patch management practices, making it less of an "ordeal" when vulnerabilities appear, through approaches like nightly container image rebuilding and ephemeral architectures.

Yam highlighted a critical distinction between traditional CVEs and malicious packages in the supply chain. He explained that a malicious package is akin to a "CVE and exploit combined," meaning the payload is already present in the environment. This necessitates a different level of triage, potentially leading directly to incident response, rather than the standard vulnerability management process, a nuance often missed by tools attempting to "have it all."

Risk Metrics Beyond CVSS

The panel critically examined traditional risk scoring. Hava acknowledged CVSS as a valuable "common language and a baseline" for severity but deemed it insufficient for a context-driven world. She introduced EPSS (Exploit Prediction Scoring System) as a crucial addition, providing insights into the "likelihood of exploitation" in the wild. She noted that EPSS distributions show "most of the vulnerability won't probably be exploited," reinforcing the need to prioritize. However, she emphasized that true risk assessment must also incorporate business impact (the consequence if a vulnerability is exploited in a specific organizational context) and reachability (whether the vulnerable code path is actually invoked). This holistic view enables a shift from "vulnerability reduction to risk reduction."

James echoed this, stating that vulnerability management often operates in a "weird place" regarding risk. He stressed that understanding "how your software is made" is paramount, recommending learning Kubernetes to grasp the underlying infrastructure and deployment processes. He argued that relying solely on CVEs can be a "crutch" that obscures this deeper understanding. Furthermore, James linked vulnerability management to a broader "onion approach" to security, where runtime mitigations like endpoint protection and firewalls historically served as the "last line of defense." He suggested that the move to cloud has "abstracted away" some of these runtime protections, making a comprehensive understanding of cloud runtime security even more critical.

Remediation Strategies

The discussion on remediation underscored its complexity and the need for more intelligent approaches. Kayla emphasized her role as a communicator, not the "fixer," highlighting the necessity of "teamwork" and strong relationships for collective problem-solving.

James advocated for a fundamental shift in how remediation is approached. Instead of treating vulnerabilities on a CVE by CVE basis, he urged organizations to think in terms of container image layers or package layers. He illustrated this with the example of a Spring framework vulnerability: sending individual CVEs to 30 different teams for independent fixes is highly inefficient. A more effective strategy would be to plan a single, coordinated "Spring migration for the organization," implemented across all teams. This package-centric view aligns better with how developers perceive and manage their dependencies, making remediation more scalable and less disruptive.

Hava introduced another layer of complexity with the "ownership" debate. She recounted an incident where a maintainer of an open-source package for improper input sanitization argued it was the developer's responsibility to sanitize inputs, not the package's. This highlights a broader issue of undefined "expectations about who's responsible of what" across the software ecosystem, extending beyond internal organizational boundaries. Yam provided a historical anecdote of a Python zip vulnerability that persisted for 15 years because the maintainer initially dismissed it as an input sanitization issue, compounded by an outdated CPE format that prevented modern scanners from detecting it – a perfect storm of ownership ambiguity and technical blind spots.

Demo / Proof of Concept

▶ Watch: James describes the executive/architect role bridging security (4:40)

This session was a panel discussion focused on conceptual and strategic challenges in vulnerability management, rather than a technical demonstration. As such, no specific demo or proof of concept was presented during the talk.

Defensive Implications

▶ Watch: Kayla explains VM operator priorities, asset management, and 'hamster wheel' (6:00)

The insights from this panel offer several actionable strategies for security teams and organizations to enhance their vulnerability management practices and reduce actual risk:

1. Strengthen Asset Inventory and Management: As Kayla emphasized, asset inventory is the foundational layer. Organizations must prioritize accurate, comprehensive documentation of all assets, including their purpose, data processed, and business criticality. Without this, effective prioritization and targeting of vulnerability remediation are impossible.
2. Shift to Context-Driven Prioritization: Move beyond generic CVSS scores. Integrate EPSS for exploitation likelihood, assess business impact, and determine reachability within the application's runtime environment. This allows security teams to focus developer effort on vulnerabilities that truly pose a risk, reducing noise and improving efficiency.
3. Foster Strong Security-Developer Relationships: Cultivate an environment of trust where developers feel safe reporting potential issues without fear of punitive action or "war rooms." Engage developers early, explain the "why" behind security requirements, and collaborate on solutions. James specifically recommended asking developers directly about potential exploitation points before even buying scanners.
4. Optimize Remediation Strategies:

• Group by Package/Framework: Instead of tracking individual CVEs, group remediation efforts around common libraries, frameworks, or container image layers. Plan coordinated upgrades or migrations to maximize developer efficiency and reduce redundant work.
• Provide Actionable Guidance: Security teams must offer clear, specific remediation instructions, including fixed versions, commit IDs, and steps to implement the fix. Don't assume developers inherently know how to fix every vulnerability.
• Clarify Ownership: Establish clear ownership guidelines for vulnerability remediation, both within the organization and when interacting with open-source maintainers, to avoid ambiguity and delays.

1. Address False Negatives Proactively: Recognize that relying on a single scanner creates blind spots. Employ a diverse set of scanning tools (SCA, SAST, DAST, IAST, runtime analysis) across the SDLC. Regularly benchmark tools and actively hunt for false negatives, especially for non-CVE-based supply chain risks like malicious packages or behavioral anomalies.
2. Invest in Deeper Visibility and Runtime Context: Implement tools that provide runtime function-level reachability and deep binary analysis to understand how vulnerabilities manifest in deployed environments. For cloud-native architectures, understanding underlying infrastructure like Kubernetes is crucial for effective risk assessment and mitigation.
3. Focus on Patch Management as a Cultural Shift: Frame vulnerability remediation as part of a continuous patch management process. Encourage practices like automated container image rebuilding and ephemeral infrastructure to make vulnerability fixes less disruptive and more integrated into routine operations.
4. Re-evaluate Security Tooling: Critically assess security tools for their ability to deliver actionable insights and facilitate risk reduction, not just high detection numbers. Demand tools that provide context, integrate well, and support remediation workflows, rather than adding to the "hamster wheel" of identification.
5. Educate and Empower Developers: Help developers understand the broader security landscape and their role in it. James's advice to "learn Kubernetes" for understanding application infrastructure is a prime example. Empowering developers with knowledge enhances their ability to identify and address security issues proactively.

Key Takeaways

• Vulnerability management is a multi-stakeholder challenge: Effective security requires harmonizing the diverse priorities and metrics of VM operators, developers, security analysts, and executives.
• Context is king for prioritization: Moving beyond raw vulnerability metadata (e.g., CVSS scores) to incorporate EPSS, business impact, and reachability is crucial for focusing efforts on truly critical risks and reducing noise.
• Shift from vulnerability count to risk reduction: Organizations should measure success by the reduction of actual business risk, not just the number of vulnerabilities identified or patched. Over-rotation on reporting numbers can be counterproductive.
• Strong security-developer relationships are foundational: Building trust and creating a "safe space" for security conversations fosters developer ownership, encourages self-reporting, and facilitates more efficient remediation.
• False negatives and supply chain risks are critical blind spots: Beyond traditional CVEs, organizations must actively identify and mitigate risks from malicious packages and other supply chain threats that often go undetected by conventional scanning.
• Remediation needs strategic overhaul: Grouping fixes by package or framework, providing clear guidance, and defining ownership are essential for making remediation scalable, efficient, and less disruptive to development workflows.

About the Speaker(s)

• Yam Yam: Currently leads the threat research team at Zscaler. Prior to this, Yam spent five years leading research at the startup Resilient and also had a brief stint leading their cloud engineering team, giving him a strong DevOps perspective. His background also includes various roles within PayPal's security organization, focusing on insider threats, threat intelligence, and applying data science and machine learning to detection. Yam served as the moderator for this panel discussion.
• Kayla Under Coffler: A lead security engineer with Zenedity, Kayla primarily represented the persona of the vulnerability management operator on the panel. Her background is deeply rooted in security engineering and analysis, where she gained years of experience running scanners, ensuring vulnerability data reached the right stakeholders, and navigating the complexities of asset management.
• Hava: As a security analyst at Sneak for over three and a half years, Hava's role involves daily triaging, tracking, and filtering potential vulnerabilities, especially in open source and container environments. She is dedicated to ensuring that vulnerabilities entered into Sneak's database are accurate, complete, and actionable. Hava also worked on Sneak's rescore feature and brought the crucial perspective of developers, emphasizing their need to move fast and the importance of contextual relevance for vulnerability fixes.
• James: The founder of Laciotech, a resource designed to help engineers find security tools. Before founding his company, James accumulated 5 to 10 years of experience in various security architecture roles. Most recently, he worked at PagerDuty, where he was involved in FedRAMP initiatives. His career also includes time at ReliaQuest and several startups, providing him with experience across different company sizes. James represented the executive and security architect role on the panel, focusing on how to bridge the communication gap between technical security teams and business leadership.