Fireproof Your Castle with Risk-First GRC

Fireproof Your Castle with Risk-First GRC

Speakers: Aakash Yadav, Lindsey Pilver

Conference: BSidesSF 2025 — April 26-27, 2025, San Francisco

YouTube: https://www.youtube.com/watch?v=QHY_Gi8N4B4

Reading time: 7 minutes

TL;DR

Most GRC programs start with compliance frameworks and work backward to risk — a sequence that reliably misses the actual threats to the business. Lindsey Pilver and Aakash Yadav from Roblox's security GRC team argue for inverting that order: identify real business risks first, then map controls and compliance requirements to those risks. The difference is not just philosophical; it changes which controls get funded and why.

Introduction

Governance, Risk, and Compliance is a field with a reputation problem. "GRC" tends to conjure images of compliance checklists, audit tick-boxes, and frameworks that generate documentation rather than security. Lindsey Pilver and Aakash Yadav, who together lead the GRC program within Roblox's security organization, opened their BSidesSF 2025 talk with a gentle acknowledgment of that reputation — and a direct argument against it.

Their thesis: the problem is not GRC as a discipline, it's the sequence. Compliance-first GRC starts with a framework (SOC 2, ISO 27001, PCI-DSS), performs a gap assessment, and calls the gaps "risks." That's backwards. Risk-first GRC starts with the business — what it does, how it makes money, what assets it depends on — asks what can go wrong, and then maps controls and compliance requirements to the actual risk landscape. Done correctly, the security controls you implement to mitigate genuine risk will satisfy 80–90% of compliance requirements anyway. The question you want leadership asking is not "Are we compliant?" but "Are we secure?"

Pilver brings a background as a statistician who moved into technology and cybersecurity via government and banking. Yadav has spent 12 years across fintech, software, and social media. Both emphasized that the data used in their examples was fabricated for illustration and does not represent Roblox's actual risk or control posture.

What a Risk Actually Is (and What It Isn't)

▶ Watch: Defining risk correctly (07:00)

Yadav opened the substantive portion with a slide listing words like "control deficiency," "asset," "threat," and "vulnerability" — and asked the audience which of these were risks. The answer: none of them. They are components of risk. The distinction matters because many GRC programs conflate these terms, producing risk registers filled with "findings" that are actually control gaps or threat categories, not actual risks.

A properly defined risk, in the team's framing, requires three elements: a threat (a person or process capable of causing harm), an asset (something with value to the business worth protecting), and an impact (an adverse consequence that would actually affect business operations). If any element is missing — if a threat actor cannot plausibly exploit a given gap to harm an asset the organization cares about — there is no risk to document. Additionally, risk is inherently probabilistic; it describes something that has not yet occurred. Once it has occurred, it becomes a finding, and findings are managed differently.

The castle metaphor running through the talk captures this well. The organization is the castle. Controls are the walls and the moat. The threat landscape attacks from all sides and evolves constantly. Understanding which directions the castle is most vulnerable is the only basis for deciding where to build the next wall.

Risk-First vs. Compliance-First: An Architecture Difference

▶ Watch: Comparing risk-first and compliance-first approaches (12:30)

In a compliance-first program, the process flows from framework to gap assessment to "risks" — which are really control deficiencies against a chosen standard. The gaps drive the remediation roadmap. The result: you get an organization optimized to pass audits, not one that has identified and prioritized its genuine exposure.

In the risk-first model, risk identification and assessment comes first. The team convenes subject matter experts, reviews the business model, identifies critical assets and processes, and asks: what can go wrong and how? Only after surfacing and documenting the top risks does the team turn to policy requirements, legal obligations, and compliance frameworks. The gap assessment still happens — but when gaps are found, they are mapped to the risks identified in the first step, creating a direct line from control deficiency to business impact.

Yadav summarized the operating principle: "I want to meet security, and I want to meet compliance as a byproduct." The maze analogy made this concrete: compliance requirements are the walls of the maze; security is the exit. You can navigate the maze by following the walls (compliance-first), or you can orient toward the exit (security-first) and find that following the walls gets you most of the way there anyway.

One structural advantage the Roblox team has: the GRC function sits within the security organization, not in legal or compliance. That reporting relationship enables the risk-first lens because it removes the pressure to prioritize regulatory requirements over operational security reality.

Qualitative vs. Quantitative Risk Assessment

▶ Watch: Qualitative and quantitative risk methods (19:00)

Pilver, the statistician on the team, gave a careful treatment of both methods and argued for using them together.

Qualitative risk assessment assigns ordinal values (e.g., 1–5 scales for likelihood and impact) to risks through expert consultation. When done systematically — using a consistent methodology, with domain experts reviewing the same criteria — it is highly effective for triaging and stack-ranking a risk portfolio. Pilver flagged one common misuse: averaging ordinal values. Ordinal scales (1, 2, 3, 4, 5) describe relative order; they do not support arithmetic. Reporting the distribution of ordinal scores using frequencies or proportions is appropriate; averaging them is not.

Cyber Risk Quantification (CRQ) expresses risk as a probable financial loss over a defined period, typically using the FAIR (Factor Analysis of Information Risk) model as its analytical framework. FAIR decomposes risk into two dimensions: Loss Event Frequency (the probability of the risk materializing) and Loss Magnitude (the financial impact when it does). A Monte Carlo simulation runs thousands of iterations across a range of input estimates — minimum, most likely, and maximum for both frequency and magnitude — producing a distribution of potential losses rather than a single number.

Pilver was precise about what CRQ does and does not do: it is an estimation methodology, not a prediction methodology. The Monte Carlo output will show that for most cyber events, the most probable outcome is no loss (infrequent but high-impact events are the characteristic pattern for cyber risk). What shifts when controls are applied is the shape of that distribution — the tail shrinks, or the mode moves left. That change in distribution is what gets presented to leadership, not a deterministic prediction.

The "false precision" warning Pilver issued is worth noting. A statistical model will produce numbers with apparent precision. Leaders tend to want predictions; risk professionals must actively manage that expectation. The CRQ output is one input to a decision, not the decision itself.

Putting It Together: A Hybrid Approach

▶ Watch: Hybrid qualitative and CRQ example (28:00)

Pilver walked through a worked example using a hypothetical top-risk list where every item initially appears equally critical. Qualitative assessment — systematic expert review against consistent criteria — produced an ordered list: one critical risk, two high risks, one medium. The critical risk: deletion of business-critical assets.

For that critical risk, the team constructed a CRQ scenario anchored on the three elements of risk: who (a malicious insider), what (misuse of authorized credentials), and what asset (critical data, resulting in significant financial and operational loss). Scoping the scenario to a specific actor type allows more precise estimates of likelihood — an insider has a materially different probability of success than an external attacker attempting the same action.

With minimum, most likely, and maximum estimates for both frequency and magnitude, the Monte Carlo produced a result showing a 46% chance of a loss event occurring, with losses ranging from roughly $3 million to $9 million when the event occurs. Two candidate controls were then evaluated: an access management control (likelihood reduction) and a SOC build-out (impact reduction). After applying the access management control, the event probability dropped to 25%. After applying the impact-reducing control, the loss range narrowed to $1.2–$4.2 million.

The important framing: these numbers do not provide the answer. They provide one data point. Leadership must weigh risk tolerance, operational friction, control cost, and the business value of the assets at risk. The GRC team's job is to give them the clearest picture of the risk landscape possible and then support the decision, not make it.

On Compliance (Yes, It Still Matters)

The talk devoted a section to compliance — and the message was not dismissive. Compliance creates trust, and trust enables business. A third-party attestation that security practices meet a defined standard is more convincing to customers and partners than an organization saying "trust us." The goal is principle-based compliance: implement the security controls that actually protect the organization, and meet compliance requirements as a natural byproduct of doing security correctly. The question "Are we SOC 2 compliant?" is less useful than "Are we actually protected against the threats that matter to our business?" — but in most industries, answering the second question will get you most of the way to answering the first.

Notable Quotes

"The question you want leadership asking is not 'Are we compliant?' but 'Are we secure?'"

"I want to implement controls that protect me from threats. And on the way to getting there, to being secure, you meet 80% to 90% of these compliance requirements."

"Our results don't point to a clear path. We are just providing decision makers with one piece of data to consider."

Key Takeaways

• Risk has three required components. A threat, an asset with value, and a plausible impact. Missing any element means the item on your risk register is not actually a risk — it's a component, a finding, or a theoretical concern.
• Invert the GRC sequence. Start with the business and identify real risks before looking at frameworks and compliance gaps. Controls implemented to address real risks satisfy the majority of compliance requirements anyway.
• Use qualitative and quantitative methods together. Qualitative assessment is the right tool for triaging and stack-ranking a risk portfolio. CRQ (using the FAIR model and Monte Carlo simulation) is the right tool for making the case for a specific control investment.
• Don't average ordinal scales. This is a statistical error that produces meaningless results. Report ordinal risk ratings using frequencies and proportions, not arithmetic means.
• CRQ is estimation, not prediction. Manage leadership expectations accordingly. A distribution of probable losses is the output — not a forecast of when an event will occur or exactly how much it will cost.