Not All Alerts Are Born Equal: Insights from AppSec Experts on Prioritizing Security Alerts
Shahar Man (Co-founder and CEO · Backslash Security), Trupti Shiralkar (Senior Engineering Manager, Product Security · Datadog), Joe Christian (Engineering Manager, DevSecOps Platforms · Paylocity)
AppSec Village @ DEF CON 31 · Day 2 · AppSec Village
Overview
In the sprawling landscape of modern software development, security teams are perpetually inundated with an "avalanche" of alerts. This panel discussion, "Not All Alerts Are Born Equal," brought together AppSec veterans Shahar Man (moderator), Trupti Shiralkar, Joe Christian, and Kiran (from eBay, filling in for Kunal from SentinelOne), to dissect the critical challenge of alert fatigue and offer pragmatic strategies for prioritization. The talk underscored a fundamental shift in application security: moving beyond mere vulnerability identification to focusing on contextual understanding, exploitability, and efficient remediation.

Key moments
- 4:00 Overcoming alert fatigue: focus on exploitability and blast radius
- 18:00 Context is king for effective vulnerability prioritization
- 21:00 Correlate vulnerabilities and empower Security Champions for triage
- 24:00 Understanding the strategic shift to Application Security Posture Management
- 27:00 Prioritizing alerts using threat intelligence, not just CVSs
- 35:00 Building developer trust and empowering them for vulnerability triage
- 43:00 Advanced correlation: using threat modeling for low/medium vulnerabilities
Not All Alerts Are Born Equal: Insights from AppSec Experts on Prioritizing Security Alerts
Speakers: Shahar Man (Co-founder and CEO, Backslash Security); Trupti Shiralkar (Senior Engineering Manager, Product Security, Datadog); Joe Christian (Engineering Manager, DevSecOps Platforms, Paylocity)
Conference: AppSec Village @ DEF CON 31
YouTube: https://www.youtube.com/watch?v=KOqMNEkGiVI
Overview
In the sprawling landscape of modern software development, security teams are perpetually inundated with an "avalanche" of alerts. This panel discussion, "Not All Alerts Are Born Equal," brought together AppSec veterans Shahar Man (moderator), Trupti Shiralkar, Joe Christian, and Kiran (from eBay, filling in for Kunal from SentinelOne), to dissect the critical challenge of alert fatigue and offer pragmatic strategies for prioritization. The talk underscored a fundamental shift in application security: moving beyond mere vulnerability identification to focusing on contextual understanding, exploitability, and efficient remediation.
The panelists, drawing from decades of combined experience, highlighted that the sheer volume of security findings from a multitude of tools often paralyzes security engineers and frustrates developers. This session was a deep dive into actionable techniques and cultural adjustments necessary to transform a chaotic alert stream into a manageable, impactful security program. It emphasized that effective AppSec is not just about detecting every flaw, but about intelligently discerning which threats truly matter to the business and how to address them collaboratively.
Why this talk matters is evident in its direct confrontation of a universal AppSec pain point. As applications grow in complexity and deployment velocity accelerates, the traditional, siloed approach to security alerts becomes unsustainable. The insights shared by Man, Shiralkar, Christian, and Kiran provide a roadmap for organizations to build more resilient and responsive security postures, ensuring that critical threats are addressed without overwhelming engineering teams or stifling innovation.
Background
▶ Watch: Overcoming alert fatigue: focus on exploitability and blast radius (4:00)
The problem of security alert fatigue is not new, but it has intensified dramatically with the proliferation of security tools and the accelerating pace of software development. As Trupti Shiralkar starkly put it, during her 18+ years in AppSec, she has used "at least 25 plus security scanners, product security testing tools," ranging from Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), Software Composition Analysis (SCA), Cloud Security Posture Management (CSPM), to Interactive Application Security Testing (IAST). This diverse toolkit, while powerful, often generates an overwhelming number of findings, many of which are false positives, low severity, or simply not exploitable in the specific application context.
Early in her career, Shiralkar experienced such severe alert fatigue that she "quit her job twice," realizing that the issue was systemic across the industry. The core problem lies in the first-generation nature of many security tools, which are "built to work" but often lack the inherent "context" of the application, its assets, or the data it protects. This context-blindness leads to a massive influx of alerts, where a high CVSS score from a scanner doesn't necessarily translate to a critical business risk.
Prior to the strategies discussed, many organizations would default to prioritizing solely based on a tool's assigned severity (e.g., "critical," "high"). However, as Joe Christian pointed out, this often leads to chasing non-critical issues while truly dangerous, context-specific vulnerabilities might be overlooked. The traditional approach also often treats security as an isolated function, with security teams "throwing tickets over the wall" to developers, leading to friction, misunderstanding, and delayed remediation. The panelists collectively highlighted that this outdated paradigm not only drains security resources but also erodes trust and collaboration between security and development teams, ultimately hindering the organization's overall security posture.
Key Findings
▶ Watch: Correlate vulnerabilities and empower Security Champions for triage (21:00)
The panel converged on several critical findings that challenge conventional alert prioritization and management, offering a more nuanced and effective approach:
- Exploitability Over Severity: The most significant finding, championed by Trupti Shiralkar, is to prioritize based on exploitability rather than just the tool's assigned severity. A vulnerability's presence doesn't equate to its exploitability. Understanding if a vulnerable function or parameter is actually invoked by the application significantly reduces the noise, potentially cutting "100% of findings down to 5%" of truly actionable items.
- Context is King: All panelists stressed that alerts must be assessed within the broader organizational context. This includes business criticality (e.g., revenue-generating applications), data sensitivity (PII, credit cards, SSNs), network exposure (public-facing vs. internal, DMZ), and the presence of existing detective or compensating controls. Joe Christian emphasized that there is "no Silver Bullet" and organizations must rationalize how general guidelines (NIST, OWASP) apply to their unique environment.
- Automation is Essential, with Empathy: Automation is crucial for managing scale, but it must be implemented thoughtfully. Kiran highlighted that initial automation efforts can create an "avalanche of alerts" for developers if not accompanied by empathy and a feedback loop. Automation should not only port findings but also make them concise and actionable for developers.
- Vulnerability Correlation: With multiple scanners often reporting the same issue, Trupti Shiralkar advocated for vulnerability correlation tools or custom engineering solutions. Consolidating duplicate findings allows teams to focus on remediation rather than redundant investigation, making the process "less overwhelming for security engineer and for developer."
- Empower Security Champions and Developers: The burden of security cannot rest solely on the security team. Kiran and Shiralkar emphasized involving Security Champions—developers with an interest in security—and training them to assist with triage and assessment. This fosters a partnership, leverages developers' deeper technical context, and helps scale security efforts, though Shiralkar cautioned that "any change takes time" and requires patience and positive reinforcement.
- Proactive Asset Management: Joe Christian underscored the fundamental importance of knowing "what's on your network." While challenging, accurate asset management (CMDB, Backstage) is foundational. Prioritizing assets based on their impact on revenue or sensitive data is key.
- The Rise of Application Security Posture Management (ASPM): Kiran and Shiralkar enthusiastically endorsed ASPM as a "major shift." It moves organizations from siloed tool outputs to an integrated, contextual, and strategic view of application security across potentially "10,000 applications." ASPM helps identify where strong remediation controls can mitigate entire categories of vulnerabilities.
- Leveraging Threat Intelligence: Joe Christian introduced the use of external threat intelligence feeds, such as SISA KEV (Known Exploited Vulnerabilities) and EPSS (Exploit Prediction Scoring System), to filter out high-severity alerts that are not actively being exploited, significantly cutting down the list of urgent issues.
- Policies, SLAs, and Persistence: While empathy is vital, establishing clear SLAs (Service Level Agreements) for vulnerability remediation and maintaining persistence in follow-ups can be effective, especially in larger organizations. Kiran mentioned a "naughty list" for teams that consistently miss SLAs without justification, which can escalate to senior leadership.
Technical Deep Dive
▶ Watch: Understanding the strategic shift to Application Security Posture Management (24:00)
The discussion provided several technical insights into how organizations can implement the key findings.
Contextual Prioritization and Exploitation Analysis:
The core technical shift is moving from a scanner's generic output to a deep, context-aware understanding of a vulnerability. Trupti Shiralkar detailed that for SAST and SCA tools, which often "blindly look for the open-source library, their version," the critical step is to determine if the vulnerable code is "actually invoking" the "vulnerable parameters, the file paths, the functions methods that are being mention in these CVEs." This requires a deeper understanding of the application's runtime behavior and code paths, going beyond static analysis to dynamic context. For example, a known vulnerability in a third-party library might only be exploitable if a specific, deprecated API call is used, which the application might not be calling at all.
Automation for Actionable Alerts:
Automation extends beyond simply integrating security tools into CI/CD pipelines. As Joe Christian highlighted, the raw output from tools is often "cumbersome" and "confusing" for developers. The technical challenge is to "make it more concise" by adding context and specific remediation guidance within the automated ticketing system. This might involve:
- Parsing and Enrichment: Automated scripts or platforms that ingest raw scanner findings, cross-reference them with asset inventory (e.g., associating a finding with a specific application, team, and data criticality).
- Template-based Remediation: Providing pre-written, context-specific remediation steps for common bug classes like SQL injection or cross-site scripting (XSS).
- Integration with Developer Workflows: Instead of security-specific dashboards, push alerts directly into developers' familiar tools like Git or their primary project management system, ensuring a "single pane of glass" for all application-specific vulnerabilities.
Vulnerability Correlation and ASPM:
The concept of vulnerability correlation addresses the technical redundancy of multiple security tools. An engineering solution, whether a commercial ASPM platform or an in-house script, can:
- De-duplication: Identify and merge identical findings reported by different SAST, DAST, or SCA tools based on parameters like code location, CVE ID, or affected component.
- Contextual Aggregation: Combine findings from various sources to build a more complete picture of a single vulnerability. For example, a SAST alert pointing to a vulnerable function, combined with a DAST finding showing it's reachable via a public endpoint, provides a higher-fidelity alert.
- Strategic Overview: ASPM tools are designed to provide a "strategy point of view" across an entire application portfolio. Technically, this involves ingesting data from all AppSec tools, mapping it to the application inventory, and providing dashboards that show coverage, risk trends, and the impact of remediation efforts across categories of vulnerabilities rather than individual findings. This allows for identifying "strong remediation control that can mitigate the entire category of a vulnerability."
Advanced Threat Intelligence and Asset Management:
Joe Christian introduced specific tools and frameworks for enhancing prioritization:
patchthis.ai: A web application that leverages SISA KEV (Known Exploited Vulnerabilities) and EPSS (Exploit Prediction Scoring System). Integrating these feeds allows teams to filter alerts based on real-world exploitation likelihood, moving beyond theoretical CVSS scores.- Asset Inventory: While difficult, tools like Backstage (Spotify's open-source developer portal) or robust Configuration Management Databases (CMDBs) are crucial. Technical efforts involve scanning network blocks, identifying assets (containers, EC2 instances), and correlating this data with application ownership and criticality. This mapping is fundamental for contextual risk assessment.
Addressing Lows and Mediums through Threat Modeling and Exploit Chaining:
The correlation of multiple low/medium severity findings into a high-impact exploit chain is a complex technical challenge. Joe Christian suggested that threat modeling is the primary mechanism. During threat modeling sessions, security and development teams should explicitly consider how seemingly minor vulnerabilities could be chained together by an adversary to achieve a significant impact. This requires understanding:
- Lateral Movement: How an attacker could move from a low-privilege access point to a more critical system using a series of weak points.
- Detection Methodologies: As Christian stated, thinking of the "network as a maze" where the adversary navigates, and the "blue team has different detection methodology" placed within. This involves ensuring robust logging and monitoring at various points in the application architecture.
- Future State (Code Provenance): Christian hinted at the next generation of security monitoring, focusing on "active exploitation at the code level." This involves code provenance, Salsa framework (Supply-chain Levels for Software Artifacts), and code notary to ensure the integrity of code from development to deployment. The detection response team would then consume this activity, flagging "this hash didn't match someone go investigate it," shifting focus from vulnerability presence to active tampering or exploitation.
Demo / Proof of Concept
▶ Watch: Building developer trust and empowering them for vulnerability triage (35:00)
The panel discussion focused on strategic and process-oriented aspects of alert prioritization and management. There was no specific demonstration or proof of concept presented during the talk itself. The panelists shared their experiences and insights from implementing these strategies within their respective organizations.
Defensive Implications
▶ Watch: Advanced correlation: using threat modeling for low/medium vulnerabilities (43:00)
The insights from this panel offer a robust framework for security teams to significantly enhance their defensive posture against application-layer threats. The key implications are:
- Shift from Quantity to Quality: Defenders must move away from the mindset of "fix everything" to "fix what matters most." This requires investing in the capability to assess exploitability and contextual risk, rather than blindly following scanner outputs. Implement processes to validate if vulnerable functions are indeed invoked and reachable.
- Invest in Contextual Prioritization Frameworks: Develop and integrate a consistent framework for evaluating alerts based on business criticality, data sensitivity (e.g., presence of PII, credit card data), and network exposure (public-facing vs. internal). This necessitates close collaboration with business stakeholders and engineering teams to classify assets and applications.
- Automate with Empathy and Precision: Prioritize automation for alert ingestion, de-duplication, and routing. However, ensure that automated alerts are concise, actionable, and include clear remediation guidance. Integrate security findings directly into developer workflows (e.g., Git, Jira) rather than separate security dashboards to reduce friction.
- Embrace Application Security Posture Management (ASPM): Security teams should explore and adopt ASPM solutions to gain a holistic, strategic view of their application security landscape. This allows for identifying systemic issues, measuring the effectiveness of controls, and making data-driven decisions on resource allocation across thousands of applications.
- Foster Security Champions and Developer Partnerships: Actively recruit, train, and empower Security Champions within development teams. Share the burden of initial triage and assessment, providing them with runbooks and quality feedback. This not only scales security efforts but also embeds security awareness directly into the development culture through positive reinforcement.
- Integrate Threat Intelligence: Leverage external threat intelligence feeds like SISA KEV and EPSS to enrich vulnerability data. This helps prioritize alerts that represent actively exploited or highly probable threats, allowing defenders to focus on real-world risks.
- Prioritize Accurate Asset Inventory: A foundational defensive measure is maintaining a precise and up-to-date asset management system (e.g., CMDB, Backstage). Knowing what applications, services, and data stores exist, and their criticality, is indispensable for effective risk assessment and incident response.
- Regularly Perform Threat Modeling: Conduct frequent threat modeling sessions, especially for major application revisions or new features. This process should involve diverse teams (dev, ops, security, privacy) to collaboratively identify potential exploit chains and design robust defensive controls, even against combinations of low and medium severity findings.
- Establish Clear Policies and SLAs: While collaboration is key, formal SLAs for vulnerability remediation, supported by leadership, provide necessary accountability. For legacy applications or persistent issues, policies for regular security reviews and forced recycling through CI/CD pipelines can help prevent technical debt from accumulating.
- Consider Purple Teaming and Code Provenance: For critical assets, engage in purple teaming exercises where red and blue teams collaborate to test defenses against specific attack paths. Looking ahead, explore code provenance, Salsa framework, and code notary to establish trust in the software supply chain and detect active exploitation at the code level.
Key Takeaways
- Prioritize Exploitability, Not Just Severity: Focus on whether a vulnerability is actually exploitable in your specific application context, rather than relying solely on scanner-assigned severity scores. This can drastically reduce alert fatigue.
- Context is Paramount: Assess alerts based on business criticality, data sensitivity (PII, credit cards), network exposure (public-facing), and existing compensating controls. A generic "critical" alert may not be a critical risk for your organization.
- Automate Intelligently and Empathetically: Implement automation for alert management in CI/CD pipelines, but ensure the output is concise, actionable, and integrated into developers' existing workflows. Empathy and clear communication are crucial to avoid overwhelming development teams.
- Empower Developers and Security Champions: Involve development teams and train Security Champions to assist with vulnerability triage and remediation. This scales security efforts, leverages their deep application knowledge, and fosters a collaborative security culture.
- Embrace Application Security Posture Management (ASPM): Adopt ASPM solutions to gain a holistic view of your application security across your entire portfolio, enabling strategic decision-making and efficient resource allocation.
- Leverage Threat Intelligence and Asset Management: Integrate external threat intelligence (e.g., SISA KEV, EPSS) for real-world exploitability insights and maintain an accurate, prioritized asset inventory as the foundation for all security efforts.
About the Speaker(s)
Shahar Man is the co-founder and CEO of Backslash Security. Prior to this, he was instrumental in building the engineering and product team at Aqua Security, where he contributed to notable open-source projects like Trivy and KubeBench. He moderated the panel, bringing his expertise in application security and startup leadership.
Trupti Shiralkar is a Senior Engineering Manager for Product Security at Datadog. With over 18 years of experience, she transitioned from a mobile developer to an AppSec practitioner. Trupti is a strong advocate for security and mindfulness, having recently returned from a 10-day Vipassana retreat. She emphasized the importance of context and exploitability in managing alert fatigue.
Joe Christian serves as the Engineering Manager for the DevSecOps Platforms team at Paylocity. He previously held leadership roles at Brex and Zappos. Joe is also pursuing a Cyber Defense PhD at Dakota State University. His contributions highlighted the importance of foundational security practices, asset management, and leveraging threat intelligence.
Kiran (last name not provided in transcript) is responsible for the security engineering function at eBay. With over 10 years in the security space, he works closely with eBay's security incident response function and has experience in AppSec and offensive security. Kiran emphasized the challenges of scale and the need for empathetic automation and strong policies.
Reviews
Dr. Zero (Offensive Security Researcher) — WEAK
This is a LinkedIn thought-leadership post stretched into a 45-minute panel. Telling a DEF CON audience in 2023 that CVSS is flawed, alert fatigue is real, and you should use EPSS isn't a revelation, it's baseline competence. Not a complete disaster, but you won't learn anything here you couldn't get from a vendor whitepaper.
Heather Calloway (CISO) — SOLID
This is a highly pragmatic panel on rescuing AppSec programs from tool-induced alert fatigue. It details the necessary shift from CVSS-based patching to prioritization based on reachability, business context, and EPSS. While it does not present net-new research, it serves as a mandatory operational baseline for any organization tired of burning engineering cycles on unexploitable flaws.