Building Effective Security OKRs

Building Effective Security OKRs

Speakers: Alex Smolen, Director of Security, LaunchDarkly

Conference: LocoMocoSec

YouTube: https://www.youtube.com/watch?v=0O6kp-qiN8I

Overview

In this insightful talk from LocoMocoSec, Alex Smolen, Director of Security at LaunchDarkly, tackles a pervasive challenge faced by security teams: demonstrating value and aligning their efforts with broader organizational goals. Smolen argues that while security professionals often find their work inherently "cool"—from exploits to secret codes—many outside the security bubble view it as scary, an unnecessary distraction, or simply incomprehensible. This disconnect leads to vague goals, misaligned efforts, and a struggle for resources.

Smolen proposes a powerful solution: Objectives and Key Results (OKRs). By adopting a structured OKR framework, security teams can transform their mission from a nebulous pursuit of "cool security things" into meaningful, measurable work that resonates with the entire organization. The talk provides a practical guide, rooted in real-world experience, on how to define impactful security objectives and track progress through verifiable key results, ultimately enabling security teams to gain buy-in, prioritize effectively, and clearly articulate their contribution to business success.

The core premise is that a security team without strategic direction is doomed to either organic, tactical improvements at best, or chaos at worst. Smolen outlines a methodology to overcome this, ensuring that security initiatives are not only technically sound but also strategically valuable and universally understood across the company. This approach empowers security to move beyond a reactive stance, fostering a proactive and integrated role within the organization.

Background

▶ Watch: Introduction and the problem with vague security goals (0:00)

The concept of Objectives and Key Results (OKRs) originated at Intel, pioneered by Andy Grove, and was famously adopted and popularized by Google through John Doerr's work, detailed in his book "Measure What Matters." An OKR system is a management methodology designed to ensure that an organization focuses its collective effort on a few critical issues. It’s structured around two core components: Objectives and Key Results.

An Objective is a significant, concrete, action-oriented, and inspirational goal. It defines what needs to be achieved. For instance, "Launch a delightful new product feature" or "Achieve market leadership in X." Crucially, objectives should be easy to understand and remember.

Key Results map these objectives to outcomes required to reach them. They define how success will be measured. Key Results must be specific, time-bound, aggressive yet realistic, measurable, and verifiable. They are quantitative metrics that indicate progress toward an objective. For example, if the objective is "Launch a delightful new product feature," a key result might be "Achieve an NPS score of 70% from early adopters" or "Increase daily active users by 20% within the first month." The talk emphasizes that Google considers hitting 70% of a key result a success, highlighting the aspirational nature of these goals.

Smolen highlights four "OKR superpowers" for organizations, as articulated by John Doerr:

1. Focus: OKRs force teams to reduce the number of initiatives, focusing on what truly matters amidst a deluge of potential problems and new technologies.
2. Alignment: They act like "turn signals," communicating where teams are headed, preventing duplicative work, and identifying gaps.
3. Accountability: Key results pinpoint whether objectives were achieved and why, fostering a culture of learning and growth.
4. Stretching: Setting tough, ambitious goals improves performance, giving teams a sense of accomplishment akin to "beating a tough video game."

The typical OKR process follows a quarterly cadence, involving planning, communication, progress measurement, and end-of-quarter assessment. While cascading OKRs (company objectives guiding team objectives, which in turn guide individual objectives) are common, Smolen stresses the importance of incorporating bottom-up ideas from individual contributors, as some of the best insights often emerge from those on the ground. For security teams specifically, the challenge is to translate the intrinsic "coolness" of security work into objectives and key results that resonate with an entire organization, many of whom may view security as a burden rather than a benefit. This talk provides the framework to bridge that gap.

Key Findings

▶ Watch: Risk: The most important concept for security objectives (10:00)

Alex Smolen’s talk distills several critical findings for building effective security OKRs, emphasizing alignment, measurability, and a pragmatic approach to risk.

First and foremost, security objectives must make sense to people outside the security team and directly support the organizational mission. Smolen recounts his experience at Clever, an education technology company handling over half of US children's personally identifiable information (PII). His breakthrough objective, "zero high severity security incidents," was simple, clear, and immediately understood by the executive team, earning the security team a prominent spot in the company's top-level OKR spreadsheet. This underscores that effective objectives translate complex security concerns into universally comprehensible business outcomes.

Second, the most important concept for aligning security objectives is risk. Smolen posits that risk is "like the physical laws of information security," bridging the gap between security professionals' desire to work on "cool security things" and the rest of the company's need to prevent "uncool security problems." Objectives must clearly demonstrate how security efforts reduce either actual risk (e.g., preventing data breaches) or perceived risk (e.g., achieving ISO 27001 and SOC 2 certifications to assure customers and enable sales). He uses the example from his time at Twitter, where prioritizing account security efforts on "high value accounts" (celebrities, advertisers) directly aligned with business impact, leading to fewer high-value compromises.

Third, effective security OKRs often involve balancing tension between different objectives. While reducing risk is paramount, it shouldn't come at the expense of other business priorities. Smolen introduced an objective at LaunchDarkly: "enable the rapid delivery of value," designed to be in tension with risk reduction goals. This ensures that security doesn't slow product teams to a grinding halt. Other balancing objectives could include improving developer experience or achieving high Net Promoter Scores (NPS) for security services, fostering a sustainable relationship with the rest of the organization.

Fourth, Key Results must be clear, measurable, and based on defined, tracked metrics. Vague key results ("process is documented," "false positive rate is improved") are ineffective because they allow for subjective interpretation. Smolen highlights the necessity of pre-work: establishing systems to measure what you intend to improve before setting key results. He provides an example of a concrete key result: "reduce the average time until identification of vulnerabilities by 50%." This requires consistent vulnerability data tracking, including type, severity (CVSS), service, team, source, and time introduced/identified/mitigated.

Finally, the process of building OKRs should be iterative, collaborative, and accountable. Smolen describes a structured process involving post-mortems of previous OKRs, bottom-up brainstorming from the team (encouraging data-backed ideas), collaboration with other teams (e.g., DevOps/IT) to establish shared objectives, and a prioritization framework considering impact, effort, confidence, and cost of delay. Crucially, OKRs are not "set and forget"; progress must be tracked publicly, and deviations should trigger re-evaluation and adaptation, allowing teams to stay agile and continuously improve.

Technical Deep Dive

▶ Watch: Building effective Key Results: clarity, metrics, input vs. output (16:40)

The "technical deep dive" in the context of security OKRs isn't about specific exploits or code, but rather the rigorous methodology and underlying data systems required to define, measure, and achieve security goals strategically. Smolen outlines a detailed framework, drawing from his experience, for operationalizing security objectives and key results.

The foundation starts with Objectives that resonate beyond the security team. Smolen advocates for objectives that are succinct and business-aligned. For example, his "zero high severity security incidents" objective at Clever was effective because it directly addressed a critical business concern. He also references GitLab's information security team's mission, which includes three key tenets: "secure the product by reducing the risk in what they ship to customers," "protect the company by reducing risk in their IT infrastructure," and "assure the customer," which focuses on reducing perceived risk to enable sales and grow revenue, rather than purely actual risk. This distinction between actual and perceived risk is crucial for diverse security objectives, such as obtaining ISO 27001 and SOC 2 certifications, which primarily serve customer assurance.

To ensure objectives are well-rounded and sustainable, Smolen suggests introducing tensioning objectives. For example, balancing "reduce risk" with "enable the rapid delivery of value." This prevents security from becoming a bottleneck and encourages the team to think like a product manager, shipping a minimum viable product (MVP) for security tools rather than cumbersome, company-wide rollouts. Objectives around developer experience or positive NPS scores for security services (e.g., after a threat modeling exercise) provide valuable feedback and foster better relationships with other teams.

The real technical rigor comes in defining Key Results. These must be unambiguous and measurable. Smolen highlights common pitfalls:

• Vague Key Results: Phrases like "process is documented" or "false positive rate is improved" are open to interpretation. Key results must be so clear that two people would unequivocally agree on whether they were met.
• Undefined Metrics: Stating "reduce vulnerabilities by 50%" is useless if there's no system to track vulnerability data consistently. Pre-work is essential to ensure measurement systems are in place.

Smolen elaborates on the importance of distinguishing between input metrics and output metrics. Output metrics directly measure the objective's outcome (e.g., time to respond to incidents). However, if an output metric is rare (like high-severity incidents), or the initiative has a delayed impact, input metrics can be used as proxies. For example, instead of just "deploying a new static analysis tool," a key result could be "at least some identified issues have been fixed" by the tool, measuring its effectiveness rather than just its deployment.

A core component of Smolen's deep dive is the practical implementation of a vulnerability tracking system. At LaunchDarkly, Jira serves as the single source of truth for vulnerability information. This requires:

1. Custom Issue Types: Standardized fields for required metadata.
2. Standardized Type: Using a consistent classification, such as a subset of the Common Weakness Enumeration (CWE), referencing HackerOne's "types of weaknesses."
3. Severity Scoring: Employing a system like CVSS.
4. Contextual Data: Tracking service and team to identify trends, and source (e.g., bug bounty, pentest, vulnerability scanners).
5. Time-based Metrics: Recording time introduced, identified, and mitigated to measure remediation speed.

To ensure data quality, Smolen emphasizes standardizing ticket titles to describe the problem and its impact (e.g., "This would allow an attacker to..."), rather than just the fix. A robust triage process is also critical for scanner-generated vulnerabilities, which often produce noise. Manual triage ensures that recorded data reflects real rather than potential risk.

With this structured data, key results become concrete. For an objective like "reduce the risk of a data breach by reducing developer-introduced vulnerabilities," a key result could be: "Reduce the average time until identification of vulnerabilities by 50%." This is measured by summing time deltas between introduction and identification, then averaging. This data can then be visualized in dashboards, linking specific initiatives (like deploying a static analysis tool) directly to top-level company goals.

Finally, Smolen delves into using risk itself as a metric. He recommends "How to Measure Anything in Cybersecurity Risk" for its quantitative approach, which advocates for measuring risk in dollars rather than abstract risk matrices (high/medium/low). This allows for meaningful comparisons and budget setting. He also references Ryan McCamy's blog posts on simple risk analysis, which describe using expert estimation and probabilistic risk assessments to forecast risk, mitigate, and then re-forecast to measure progress. An example scenario: "an adversary accessing production from a developer laptop in Q3," with initial and end-of-quarter forecasts forming the baseline and progress measurement. Documenting how security work ties to risk, whether quantitatively or qualitatively, is critical for effective OKRs.

Demo / Proof of Concept

▶ Watch: Practical example: Tracking vulnerabilities for Key Results (20:40)

While Alex Smolen's presentation does not feature a traditional technical demonstration of a tool or an exploit, the entire talk serves as a comprehensive "proof of concept" for his methodology. The "demo" is the detailed, step-by-step process he outlines for building and implementing effective security OKRs, validated by his experiences at companies like Clever and LaunchDarkly.

Smolen uses concrete examples throughout the talk to illustrate how his framework works in practice:

• Clever's "zero high severity security incidents" objective: This demonstrates how a clear, business-focused objective can gain executive buy-in and align an entire organization.
• LaunchDarkly's vulnerability tracking in Jira: He details the custom fields, standardization of vulnerability types (referencing HackerOne/CWE), severity scoring (CVSS), and time tracking (introduced, identified, mitigated). This showcases a practical, data-driven approach to measuring security posture.
• The "reduce average time until identification of vulnerabilities by 50%" key result: This provides a specific, measurable example of how a team can set an aggressive yet achievable target, directly linking a security initiative (like implementing a static analysis tool) to a reduction in risk.
• GitLab's transparent mission and LaunchDarkly's balancing objectives: These illustrate how to craft objectives that address both actual and perceived risk, and how to introduce tension to ensure security enables, rather than hinders, business velocity.

Furthermore, Smolen describes the team's internal process for building OKRs, which functions as a practical demonstration of his principles: the post-mortem of previous quarters, bottom-up brainstorming, cross-team collaboration, and a prioritization framework considering impact, effort, confidence, and cost of delay. This entire operational flow is the "proof" that his OKR strategy is not merely theoretical but a workable, actionable blueprint for security leaders.

Defensive Implications

▶ Watch: Team process for building effective security OKRs (28:00)

The defensive implications of Alex Smolen's talk are profound, shifting the focus from purely technical defensive measures to a strategic framework for managing and articulating security efforts. For security leaders and practitioners, adopting an effective OKR system can dramatically enhance their defensive posture in several ways:

1. Strategic Alignment and Prioritization: By aligning security objectives with top-level company goals and expressing them in terms of risk reduction (both actual and perceived), security teams can ensure their efforts are focused on what truly matters to the business. This prevents random, tactical work and instead prioritizes initiatives that have the highest impact on reducing business-critical risks or enabling revenue growth (e.g., through compliance like ISO 27001 and SOC 2).

1. Improved Resource Allocation and Buy-in: When security goals are clear, measurable, and tied to business outcomes, it becomes easier to justify requests for budget, headcount, and other resources. Demonstrating a clear return on investment or risk reduction in tangible terms (e.g., reducing average time to identify vulnerabilities by 50%) helps secure the necessary support from leadership and other departments.

1. Enhanced Accountability and Continuous Improvement: Well-defined key results with clear metrics foster accountability within the security team. Regular tracking and public assessment of progress (and failures) create a learning loop. If a key result isn't met, the team can analyze why—was the initiative flawed, the target too aggressive, or the measurement incorrect? This iterative process drives continuous improvement in defensive strategies and execution.

1. Better Cross-Functional Collaboration: The emphasis on collaborating with other teams (DevOps, IT, product) to establish shared objectives is crucial. Security often acts as a gatekeeper; by thinking like a product manager and seeking to "enable the rapid delivery of value" alongside risk reduction, security becomes a partner rather than an impediment. This can lead to more proactive security integration into the SDLC, infrastructure, and business processes, reducing vulnerabilities at the source.

1. Data-Driven Decision Making: Implementing robust vulnerability tracking systems, like using Jira as a single source of truth with standardized metadata (CWE, CVSS, time metrics), provides the data necessary to understand the security posture. This data informs where to focus defensive efforts, identify trends (e.g., which teams or services introduce the most vulnerabilities), and measure the effectiveness of specific defensive tools (e.g., static analysis tools).

1. Effective Risk Management: Moving beyond abstract risk matrices to a quantitative understanding of risk (e.g., measuring risk in dollars as suggested by "How to Measure Anything in Cybersecurity Risk") allows defenders to compare cybersecurity risks to other business risks. This facilitates more informed decisions on where to invest defensive capital and how to communicate risk to non-technical stakeholders. Using expert estimation and probabilistic risk assessments for specific scenarios provides a structured way to measure and track risk reduction.

In essence, Smolen's talk provides a blueprint for security teams to become more effective, transparent, and integrated components of their organizations, ultimately leading to a more resilient and defensible enterprise.

Key Takeaways

• Align Security with Business Outcomes: Security objectives must be clear, understandable to non-security personnel, and directly support the company's mission. Frame security goals in terms of business value, such as "zero high severity security incidents" or enabling revenue through customer assurance.
• Risk is the Unifying Language: Utilize the concept of risk (both actual and perceived) to align security efforts with organizational priorities. Quantify risk where possible, moving beyond subjective risk matrices to measurable impacts, potentially in financial terms.
• Balance Objectives for Sustainability: Introduce "tensioning objectives" (e.g., "reduce risk" vs. "enable rapid delivery of value" or "improve developer experience") to ensure security acts as an enabler, not a blocker. This fosters better relationships and integrated security practices.
• Key Results Must Be Measurable and Specific: Avoid vague key results. Define clear, quantifiable metrics, and ensure systems are in place to track them before setting targets. Distinguish between input metrics (project completion) and output metrics (objective achievement), using both as appropriate.
• Implement a Robust Vulnerability Tracking System: Establish a single source of truth (e.g., Jira) for vulnerability data, complete with standardized fields (type, severity, service, team, source, time introduced/identified/mitigated). This data is crucial for setting and measuring key results like reducing vulnerability identification or remediation times.
• Embrace an Iterative and Collaborative OKR Process: Regularly review and assess past OKRs, encourage bottom-up idea generation, and collaborate with other teams to build shared objectives. Prioritize based on impact, effort, confidence, and cost of delay, and be prepared to adjust OKRs as circumstances change.

About the Speaker(s)

Alex Smolen is the Director of Security at LaunchDarkly. His career journey has provided him with diverse and valuable experience in leading security initiatives. Prior to his current role, Smolen served as the security lead at Clever, an education technology company responsible for handling a significant portion of US children's personally identifiable information. In this capacity, he was instrumental in defining security strategy and measuring its effectiveness, navigating a strong organizational culture of OKRs. Before Clever, Smolen was part of the original security team at Twitter, where he was responsible for account security during a period when account compromises were a significant concern. His experience spans from hands-on security challenges at a large social media platform to strategic security leadership in critical data environments and, now, directing security for a feature management platform. He is a proponent of using structured methodologies like OKRs to align security efforts with broader business objectives and communicate their value effectively.