UNFAIR after all; Critical considerations for risk management frameworks

UNFAIR after all; Critical considerations for risk management frameworks

Speakers: Jack Burgess

Conference: Blue Team Con Online

YouTube: https://www.youtube.com/watch?v=6uYtLgJakdc

Overview

In this insightful Blue Team Con Online presentation, Jack Burgess challenges conventional wisdom in cybersecurity risk management, arguing that many widely adopted frameworks are fundamentally ill-equipped to handle the existential risks that truly threaten an organization's survival. Moving beyond the often-misleading precision of current scoring systems, Burgess advocates for a paradigm shift towards robust, interpretable models focused on preparedness and increasing "survival time" rather than mere prediction.

The talk is designed for a broad audience across the security space, from operations and application security to budget allocation, aiming to provide a fresh perspective on how organizations can derive maximum value from their security investments. Burgess contends that by understanding the true nature of risk—particularly its long-tailed, non-Gaussian distribution—and adopting simpler, more reliable approaches, companies can navigate the complex threat landscape more effectively and avoid becoming the next high-profile casualty of a catastrophic security event.

Burgess, drawing on a decade of experience in physics, SecOps, risk, and machine learning, uses compelling real-world case studies to illustrate the catastrophic failures that occur when security teams fail to correctly assess and prioritize threats. His presentation provides a critical examination of popular frameworks like CVSS and FAIR, highlighting their limitations and proposing practical, statistically grounded alternatives that align security efforts with genuine business resilience and customer trust.

Background

▶ Watch: Introduction and scope of the talk (2:29)

The premise of Burgess's talk is rooted in the observation that despite the proliferation of cybersecurity frameworks and sophisticated tools, organizations continue to suffer devastating, often existential, security incidents. These incidents expose a critical gap in how risk is traditionally understood and managed. Burgess argues that while frameworks like PCI DSS and SOC 2 are essential for compliance and basic hygiene, they are often designed by non-security experts (e.g., accountants) and lack the deep security insight needed to confront truly business-ending threats.

Burgess illustrates this problem with three stark real-world examples:

1. Maersk (2017 NotPetya Attack): The shipping giant, with approximately 100,000 employees, was hit by the NotPetya "ransomware" that was, in fact, purely destructive. The attack wiped out their entire Active Directory forest, crippling operations globally. Wired magazine described it as "the most devastating attack in history." Maersk narrowly avoided total collapse only due to a single, uncompromised Active Directory server in a remote office, which had to be physically transported back to HQ. Burgess uses this to highlight the disconnect between potential loss (the entire value of Maersk) and realized loss (significant, but not fatal, due to an improbable recovery). Traditional threat modeling, he argues, often underestimates the true potential for total loss.

1. Travelex (2019-2020 Ransomware): The currency exchange company, with about 10,000 employees, fell victim to a serious ransomware attack exploiting unpatched VPN vulnerabilities. This led to massive customer distrust and, coupled with the pandemic, ultimately forced the company into administration. Travelex serves as an example of dose response—a single incident might be survivable, but a series of hits, especially when compounded by other factors, can be fatal. The company, though still existing under new ownership, operates at a significantly reduced scale.

1. MedicSecura (Australian Data Breach): This Australian government-backed healthcare provider experienced a breach affecting half of Australia's population. Despite prior assurances of robust security, the incident led to immense public backlash and reputational damage. MedicSecura highlights how loss of trust, even for a government-regulated entity with limited competition, can lead to forced market exit. Burgess emphasizes that had they invested "a little bit more security at the right time," the outcome would have been vastly different, implying an almost infinite return on that marginal investment post-breach.

These cases underscore a fundamental flaw: existing risk management tools often fail to account for these "big existential impacts." Burgess then critically examines several popular frameworks:

• CVSS (Common Vulnerability Scoring System): Used for categorizing vulnerability severity (0.0-10.0). Burgess criticizes its "oodles of precision" without corresponding accuracy, lack of error bars, non-repeatability due to subjective scoring, and compressed score ranges that make differentiation difficult.
• FAIR (Factor Analysis of Information Risk): Measures risk in dollars, breaking it down into loss event frequency and magnitude. While seemingly robust, Burgess points out that FAIR explicitly advises against considering losses greater than the company's value, directly contradicting the real-world scenarios of Maersk, Travelex, and MedicSecura. It fails to account for existential risk, which security engineers intuitively understand as potentially infinite.
• EPSS (Exploit Prediction Scoring System): Predicts the likelihood of a vulnerability being exploited within 30 days, incorporating CVSS scores, vendor information, and exploit availability. Burgess argues EPSS is a lagging indicator and, crucially, a prediction system. Predictions are inherently fallible, and the cost of being wrong about a critical vulnerability can be catastrophic, outweighing the benefits of efficient prioritization for less severe threats.

Other approaches like Impact Grids are praised for their speed, repeatability, and ability to handle existential risk, but are criticized for a lack of differentiation. Risk Matrices are noted for their potential traps and often incorrect implementation. Burgess also dismisses the idea that complexity equals better, citing research that debunks this notion. Finally, he warns against relying solely on red team findings without a robust risk management framework, as deconfliction processes can inadvertently obscure the true rate of detection and the likelihood of actual incidents.

Key Findings

▶ Watch: Travelex Ransomware: Unpatched vulnerabilities lead to administration (6:07)

Burgess's talk distills several critical findings that challenge traditional cybersecurity risk assessment:

1. Existential Risk is Paramount: The most significant risks are those that threaten the very survival of the organization. Traditional frameworks often fail to adequately model or prioritize these "black swan" or "long-tail" events, leading to a false sense of security. The focus should shift from managing average daily risks to preventing catastrophic failures.
2. Power Law Distributions Govern Security Incidents: Unlike many natural phenomena that follow a Gaussian (bell curve) distribution, security incidents, like wealth or botnet size, adhere to a power law distribution. This means that extremely rare, high-impact events in the "long tail" contribute disproportionately to the overall risk and are far more likely to occur than a Gaussian model would suggest. Organizations are not just dealing with slightly larger-than-average incidents; they face potentially infinitely large ones.
3. Prediction is Inherently Flawed and Costly: Relying heavily on predictive models (like EPSS) for vulnerability prioritization is problematic. Security environments are characterized by concept drift (constant change in underlying systems and threats) and small numbers of truly catastrophic incidents, making robust statistical prediction difficult. Furthermore, the cost of a false negative (failing to predict a critical exploit) can be company-ending, overshadowing the efficiency gains from accurate prediction of minor threats.
4. Robustness Over Precision: In highly variable and rapidly changing environments, simple, robust, and interpretable models outperform complex, "precise" ones. The Rashomon effect suggests that in high-variance situations, multiple different models can yield similar predictive power, making simplicity and clarity more valuable than intricate algorithms that are hard to understand or replicate.
5. Dose Response is Non-Linear: The impact of a security incident does not scale linearly with its size. An attack affecting 2,000 laptops is not merely 100 times worse than one affecting 20; it might represent an entirely different, existential threat due to the non-linear nature of system failure and recovery capabilities.
6. Ergodicity and Absorbing Barriers: A company's experience of repeated incidents is not equivalent to a consultant observing many companies at once. This ergodicity problem means that a single company faces absorbing barriers—events that can end its existence. The focus for risk management should therefore shift from quantifying "loss" to maximizing survival time, or the number of incidents an organization can withstand before hitting an absorbing barrier.
7. Security Drives Trust and Revenue: Beyond mere protection, security is a direct contributor to business value by fostering customer trust. In established markets where product differentiation is minimal, a reputation for strong security can be the primary source of competitive advantage and revenue growth. Conversely, a major breach can irrevocably destroy this trust, as seen with MedicSecura.

Technical Deep Dive

▶ Watch: MedicSecura Data Breach: Impact on a government-backed business (7:52)

Burgess constructs his argument on a foundation of economic and statistical principles, challenging the prevailing technical approaches to risk management. He defines risk fundamentally as "dollars per year," directly linking it to an organization's risk appetite—the maximum acceptable dollar loss per year. This perspective immediately frames security as an investment with a measurable return on investment (ROI). A well-functioning SOC, for instance, should yield a positive ROI, but a poorly designed one, with highly privileged accounts or insufficient controls, can generate negative value by creating new vulnerabilities or slowing down engineering teams.

The concept of value is critical; it's what customers pay for. Security contributes to this value not just by preventing losses but by building trust, which directly translates to revenue. In mature markets, where product innovation might be saturated, trust becomes the primary differentiator. Burgess contrasts Apple, which builds trust through robust security reporting and features, with MedicSecura, which lost its market position due to a catastrophic breach and subsequent erosion of trust. He also distinguishes between potential loss (the unmitigated, often unknowable, maximum impact) and realized loss (the actual cost after mitigations), emphasizing that threat modeling often underestimates the former.

Burgess outlines various modes of risk management that organizations typically engage in, often simultaneously or as they mature:

• Business as Usual: Handling daily alerts and common risks. High investment, but low impact on existential threats.
• Compliance Obligations: Meeting regulatory requirements (e.g., PCI, SOC 2). Low investment, but essential for market access, not necessarily for advanced security.
• Preventing Catastrophe: Focusing specifically on the "tails" of the risk distribution to stop company-ending events. High investment, high ROI.
• Trust Revenue: Proactively building security into products and processes to enhance customer trust and competitive advantage. Potentially enormous ROI.
• Terminal Presentation: Working backward from the hypothetical scenario of explaining a major breach to Congress, ensuring all "reasonable" measures were taken. This approach fosters proactive, defensible security decisions.
• Security Hygiene: Implementing fundamental controls like firewalls, patching, and essential security frameworks (e.g., ASD Top 8). High ROI initially, but with diminishing returns after a certain point.

The statistical core of Burgess's argument lies in the distinction between Gaussian (bell curve) distributions and power law distributions. He explains that phenomena with no relationship between events (like human height) tend to be Gaussian, where extreme values are rare. However, security incidents, like wealth or botnet size, exhibit autocorrelation; the size of an existing botnet, for example, makes it easier to expand. This leads to a power law distribution, characterized by a "narrow trunk and a very long tail," where extreme events are far more probable and impactful. Burgess graphically demonstrates this with "flower bed" simulations, showing how a power law distribution can produce events "way off the page" that are never seen in Gaussian models.

Burgess highlights several challenges to effective prediction in security:

• Bayes' Law: Even with "good" detection systems, a high volume of benign events means alerts will still miss a lot of serious incidents and generate many false positives, especially if the underlying rate of true serious incidents is low.
• Concept Drift: The underlying security landscape (new printers, firewalls, auth methods) constantly changes, meaning past incident data quickly becomes irrelevant, preventing convergence to a stable distribution.
• Small Numbers: The relatively low frequency of truly catastrophic security incidents means that statistical laws like the Central Limit Theorem (which applies to averages) and the Law of Large Numbers (which requires many trials) do not apply to the tails of the distribution, which are precisely what security practitioners care about.
• Most Intrusions Aren't Seen: Citing research (e.g., David J. Bianco's work), Burgess notes that a vast majority of malware and intrusions are only detected in one environment, indicating a massive unseen space of threats.

He introduces dose response to explain the non-linear impact of incidents, using the analogy of dropping a coffee mug: dropping it 20 times from a table might not break it, but dropping it once from 20 times the height (a roof) almost certainly will, highlighting that impact is not linearly proportional to "dose."

Finally, Burgess addresses ergodicity and absorbing barriers. A consultant, observing many companies at once, might see a "Gaussian" distribution of outcomes. However, an individual company experiences a single, non-ergodic path where a catastrophic event can lead to an "absorbing barrier"—a point of no return where the company ceases to exist. This shifts the focus from merely measuring loss to maximizing survival time—how many "jumps" (incidents) a company can withstand before hitting the cliff.

Demo / Proof of Concept

▶ Watch: The punchline: Limitations of PCI, SOC 2 frameworks (10:04)

While the talk did not feature a live, interactive demo of a specific tool, Jack Burgess presented a conceptual simulation to illustrate the effectiveness of his proposed improper linear model approach to risk scoring. He described a hypothetical scenario where 10 binary factors (variables) contribute to a risk score, with each factor being either present (1) or absent (0). The sum of these factors provides a simple, unit-weighted risk score.

Burgess simulated generating a large number of "alerts" or scenarios, where the presence of each of the 10 factors was random. He then analyzed the density of the resulting scores (0-10). The key observations from this simulation were:

1. Rarity of High Scores: Even with random factor presence, scores of 8, 9, or 10 were infinitesimally rare. This demonstrates that a simple additive model naturally filters out the "noise" of everyday, less critical events. The "stuff that we do care about is pretty small," making it manageable.
2. Increased Precision for High Scores: Burgess argued that if a real, serious incident involves many of these factors (as would be defined by organizational experience and after-action reviews), then a high score from this model is "more likely to be something serious." The odds of unrelated factors randomly aligning to produce a high score for a fluke event are "very small," thereby increasing the precision of the high-score alerts.
3. Manageability of the "Tail": The simulation visually conveyed that the number of truly high-risk events (scores of 8-10) is very small. This makes the "tail" of the risk distribution manageable, allowing security teams to focus their resources on the few events that genuinely threaten survival, rather than being overwhelmed by a flood of low-impact alerts.

This conceptual proof of concept effectively demonstrated that a simple, additive scoring model, even with random inputs, can effectively differentiate between low-impact noise and potentially critical events, aligning with the goal of increasing precision and focusing on the most severe risks.

Defensive Implications

▶ Watch: Beginning the analysis of existing security frameworks: CVSS (11:09)

The insights presented by Jack Burgess offer several critical implications for cybersecurity defenders seeking to build more resilient and effective security programs:

1. Prioritize Existential Risks Above All Else: Defenders must explicitly identify and focus on threats that could lead to the total collapse or irreversible damage of the organization. This means moving beyond compliance-driven or average-case risk assessments and actively modeling "what if everything goes wrong?" scenarios.
2. Shift from Prediction to Preparedness and Robustness: Instead of chasing elusive predictive accuracy, security teams should focus on building robust defenses and incident response capabilities that prepare them for when a catastrophic event occurs, not if. The goal is to increase the organization's "survival time" by making it more resilient to high-impact incidents.
3. Adopt Improper Linear Models / Predictive Checklists for Risk Scoring: This is the core actionable recommendation. Defenders should implement simple, additive models for assessing risk:

• Select 10-15 key, objective factors: These factors should be readily identifiable and agreed upon by the security team as indicators of serious risk (e.g., "critical environment breached," "exploit code publicly available," "unpatched critical vulnerability").
• Ensure objectivity and repeatability: Each factor's presence or absence should be determinable in the same way by any analyst, minimizing subjectivity. This can be achieved by defining clear thresholds (e.g., "more than two indicators of compromise," "more than 50% of endpoints affected").
• Avoid linear scaling for incident counts: Recognizing dose response, do not simply multiply a risk score by the number of affected systems. Instead, use breakpoints: define separate factors for "1 system affected," "more than 1 system affected," "more than 10 systems affected," to reflect non-linear increases in impact.
• Automate where possible: The simplicity of these models makes them ideal for automation, providing rapid, consistent risk scores during incidents.
• Use for training: These checklists serve as an excellent tool for onboarding new security personnel, quickly conveying what the organization truly cares about from a risk perspective.

1. Understand and Plan for Power Law Distributions: Defenders must internalize that security incidents are not normally distributed. This means rare, high-impact events are more probable and devastating than often assumed. Defensive strategies should therefore focus on hardening against these "long-tail" risks, even if they seem unlikely in a Gaussian world.
2. Leverage Security for Trust and Revenue: Position security not just as a cost center, but as a direct contributor to business value. By building a reputation for strong security and transparency, organizations can enhance customer trust, differentiate themselves in the market, and ultimately drive revenue. This requires proactive communication and demonstrable security practices.
3. Optimize Security Hygiene Strategically: While essential, recognize that security hygiene (e.g., patching, firewalls) has diminishing returns. Once a baseline of effective hygiene is established, additional investment should shift towards preventing catastrophe and building trust, rather than pouring more resources into incremental hygiene improvements.
4. Be Skeptical of Overly Complex Models: Resist the urge to adopt complex, black-box predictive models that are difficult to understand, validate, or adapt to concept drift. Prioritize models that are transparent, interpretable, and robust, even if they appear less "precise" on paper.

By implementing these implications, security teams can move away from reactive, compliance-driven approaches towards a more proactive, survival-oriented strategy that genuinely protects the organization from its most significant threats.

Key Takeaways

• Existential risks are the most critical threats, often overlooked by traditional risk management frameworks that fail to account for catastrophic, company-ending events.
• Security incidents typically follow a power law distribution, meaning rare, high-impact events in the "long tail" are more probable and have disproportionate consequences than commonly assumed.
• Prediction in security is inherently flawed due to concept drift and small numbers of true incidents; a focus on robustness and preparedness for "when," not "if," is more effective than trying to predict "what" or "when."
• Improper linear models (simple, additive, unit-weighted checklists of 10-15 objective factors) are a superior, more robust, and interpretable method for risk scoring in volatile security environments.
• Security contributes directly to business value and revenue by building customer trust; conversely, a major breach can irrevocably destroy this trust and lead to market exit.
• Defenders should prioritize maximizing survival time by focusing on non-linear dose response and designing objective, repeatable risk factors that differentiate true catastrophic threats from everyday noise.

About the Speaker(s)

Jack Burgess is a seasoned security professional with approximately a decade of experience across various domains including physics, SecOps, risk management, and machine learning. He is associated with Triangle Way Security, and his expertise lies in developing practical, statistically grounded approaches to cybersecurity challenges. Burgess is passionate about helping organizations move beyond conventional frameworks to build more resilient and effective security programs.