Imperfect Security: Doing Less to Achieve Better Security

Imperfect Security: Doing Less to Achieve Better Security

Speakers: Kevin Hanaford

Conference: BSidesSF 2024

YouTube: https://www.youtube.com/watch?v=EAL1NiKU4Mk

Overview

In his BSidesSF 2024 talk, "Imperfect Security: Doing Less to Achieve Better Security," Kevin Hanaford, Trust Engineering Lead at Discord, challenges the conventional pursuit of "perfect security." Hanaford argues that perfect security is an unattainable ideal, leading to inefficiency, high costs, and ultimately, frustration. Instead, he advocates for an "imperfect security" philosophy, which focuses on operating within reality, maximizing efficiency, prioritizing the human element, and fostering collaboration. This approach, he contends, allows security teams to achieve better, more sustainable outcomes by strategically allocating resources and building stronger relationships across the organization.

Hanaford's presentation is a strategic and philosophical deep dive into how security professionals can shift their mindset from an all-encompassing, often futile, quest for perfection to a pragmatic, impact-driven methodology. He draws on principles like the Pareto principle and insights from human psychology to illustrate why a less-is-more approach can yield superior results. The talk is particularly relevant for security leaders and practitioners grappling with limited budgets, staffing constraints, and the ever-evolving threat landscape, offering a refreshing perspective on how to build effective security programs that truly enable the business.

The core message is that by accepting the inherent imperfections of security and focusing on high-impact, human-centric, and collaborative strategies, teams can move beyond the "department of no" stereotype and become true business enablers. Hanaford's insights are not just applicable to security but broadly to any function striving for excellence in a resource-constrained and dynamic environment, emphasizing the importance of adaptability and a positive, proactive approach.

Background

▶ Watch: Defining 'perfect security' as an unattainable ideal. (1:00)

The concept of "perfect security" often serves as an aspirational goal within the industry, yet Hanaford quickly dismisses its feasibility. He paints a picture of this elusive ideal: a company with no vulnerabilities, employees immune to phishing, least-privileged, role-based, attribute-based permissions that are consistent everywhere, a staging environment that perfectly mirrors production, a Data Loss Prevention (DLP) system that works flawlessly, and Chrome always up to date. He humorously notes that achieving even one of these, like consistently updated Chrome, could likely defeat the entire premise, rendering a security team either "very rich or very unemployed."

The common adage, "perfect is the enemy of good," forms the philosophical bedrock of Hanaford's argument. This principle, he explains, is consistent across various industries because the pursuit of perfection demands immense energy and often yields diminishing returns, whereas "good can be good enough." He traces this idea back to the Pareto principle, or the 80/20 rule, popularized in the early 1900s by Vilfredo Pareto, who observed that 80% of Italy's land was owned by 20% of the population. This principle, which suggests that 20% of efforts account for 80% of results, has since been observed in manufacturing, software engineering, and real estate. Hanaford clarifies that the percentages are not rigid and don't necessarily sum to 100%, but the core idea of disproportionate impact holds.

Further illustrating this imbalance, Hanaford references Steve Ballmer's 2002 "Trustworthy Computing" memo, which highlighted that 20% of all bugs caused 80% of all errors. Ballmer further refined this, noting that a mere 1% of bugs could cause half of all errors, underscoring the critical importance of identifying and addressing the most impactful issues rather than chasing every single flaw. Another related concept is Tim Cargo's 90/90 rule from the 80s and 90s, which posits that "the first 90% of code accounts for the first 90% of development time, and the remaining 10% of code accounts for another 90% of development time." This tongue-in-cheek rule perfectly encapsulates the exponential effort required to achieve the final, often marginal, increments of perfection.

Hanaford concludes that perfect security is not only impossible but also inefficient, carries a high cost, has a skewed Return on Investment (ROI), and is "potentially only good for the memes." He jokingly suggests that the only way to achieve perfect security is to "delete your data, turn off your servers, fire everybody, and move to a mountain where you don't have cell phone reception." This sets the stage for his alternative: imperfect security, a pragmatic framework designed to achieve better security outcomes by acknowledging and working within real-world constraints.

Key Findings

▶ Watch: Outlining the four pillars of 'imperfect security': Reality, Efficiency, Peop... (5:30)

Hanaford's talk outlines four core pillars that define and enable imperfect security: operating in reality, being efficient, focusing on people, and embracing collaboration. Each pillar addresses a fundamental challenge in security and offers a strategic shift in approach.

1. Operating in Reality:

• Infinite Work, Finite Resources: Security work is endless, but teams always face limitations in time, money, and people. Humans are adept at generating new problems, and resources are consistently constrained, especially for managers dealing with tight budgets and lower headcounts.
• Unregulated Attackers: Attackers operate with far fewer rules and regulations than defenders, giving them an inherent advantage. Defenders must contend with legal, compliance, and ethical boundaries that attackers disregard.
• Ephemeral Solutions: Nothing in security works forever. Solutions require continuous iteration and improvement, as environments and threats evolve. The shift to distributed workforces post-2020, for example, was an unforeseen change that rendered many office-centric security plans obsolete.
• Business as Priority: The business's primary goal is to make money and achieve its objectives. Security, while critical, is unlikely to ever be the absolute number one priority. This means security teams must align with business goals.
• Budget Constraints: Security budgets are often limited. Hanaford cites statistics: an average of 11% of IT budget across US and European organizations is dedicated to security, which is roughly 1% of a company's total annual budget. Furthermore, 60% of companies reduced security budgets due to economic downturns, and two-thirds of US organizations reeled back security hiring in 2024.

1. Being Efficient:

• Wasted Energy: Energy is frequently wasted on the wrong things, such as being solution-oriented (buying tools without fully understanding the problem), focusing on cool, new, or novel technologies (leading to premature adoption without foundational work), or getting ahead of ourselves (implementing solutions before the organization is ready).
• Not Invented Here Syndrome: This tendency for a project group to believe it possesses a "monopoly of knowledge" leads to the rejection of external ideas, impeding innovation and hindering external and internal relationships. Learning from peers and industry progress is crucial.
• Technical Debt: Neglecting technical debt slows down teams significantly. Languishing dependencies, for instance, can turn into massive, time-consuming remediation efforts when a zero-day vulnerability emerges.

1. Focusing on People:

• Human Element in Attacks: Almost all attacks involve humans. The Verizon Data Breach Investigations Report (DBIR) indicates that 68% of breaches involved a human element. Social engineering, particularly pretexting, phishing, and extortion, accounts for 96% of breaches related to social engineering, exploiting human psychology.
• Human Behavior: People generally want to do the right thing but may not know how. Convenience often outweighs process, especially when processes are high-friction. Retraining habits is challenging, taking an average of 66 days to form a new habit, and even longer to unlearn old ones.

1. Embracing Collaboration:

• Teamwork Benefits: Groups consistently outperform individuals, even when the group comprises less experienced members than a highly skilled individual. Teamwork also leads to happier employees and benefits significantly from differing perspectives, fostering better problem-solving.
• "Department of No" Perception: The security industry has often cultivated a perception of being the "department of no," constantly rejecting ideas. This leads to frustration, avoidance of the security team, and a tendency for people to "ask for forgiveness, not permission."
• High-Friction Controls & Dumb Policies: Overly burdensome controls and poorly explained policies (like some VPN mandates) stress people out, hinder productivity, and encourage workarounds, ultimately undermining security goals.

These findings collectively form the foundation for Hanaford's imperfect security framework, emphasizing a pragmatic, human-centric, and collaborative approach to security program management.

Technical Deep Dive

▶ Watch: Actionable advice for 'operating in reality': minding business priorities and... (11:00)

While Hanaford's talk is not a deep dive into specific code or protocols, it offers a robust framework for applying strategic and operational principles to technical security programs. The "imperfect security" philosophy provides a meta-technical approach, guiding how security teams should design, implement, and manage their technical controls and processes.

Operating in Reality: Strategic Resource Allocation and Risk Management

In a technical context, acknowledging the "infinite amount of work" and "finite resources" means security teams must be highly strategic about where they invest their technical efforts. This involves:

• Mind Your Business: Understanding the core business functions and revenue streams is paramount. Technical security initiatives should directly support these, rather than existing in a vacuum. For example, prioritizing the security of critical customer-facing applications over internal, low-impact tools, even if the latter has more vulnerabilities.
• Know Your Risks: Technical teams must identify, categorize, and track risks associated with unaddressed security issues. This isn't just about vulnerability scanning; it's about understanding the business impact of a potential exploit. Framing a lack of laptop encryption as a multi-million dollar expense if customer data is compromised, rather than just a compliance checkbox, is a technical risk articulated in business terms. This allows for better justification of technical investments, such as endpoint security solutions or data encryption at rest and in transit.
• Plan for Exceptions: No technical control or policy will be universally applicable. Security architects and engineers must design systems that can accommodate legitimate exceptions without compromising overall security. This means building robust exception management frameworks, ensuring that deviations from standard configurations (e.g., specific firewall rules, elevated access for critical operations) are documented, reviewed, and monitored as known risks.

Being Efficient: Goal-Oriented Technical Implementation

Efficiency in a technical security program means moving beyond reactive "whack-a-mole" security to a proactive, outcome-driven approach:

• Be Goal-Oriented: Instead of simply deploying a new tool (solution-oriented), technical teams should define desired outcomes. For instance, the goal might be "access management is easy, consistent, and transparent" rather than "implement a new Identity and Access Management (IAM) solution." This allows for a more flexible and effective selection of technical solutions, which might involve integrating existing systems, developing custom scripts, or adopting a new platform.
• Avoid Reinvention: This is a critical technical principle. Instead of building custom solutions for common security problems, technical teams should leverage existing open-source tools, industry standards, or commercial off-the-shelf (COTS) products. For example, rather than developing a proprietary vulnerability scanner, integrate established tools like OWASP ZAP or commercial offerings. Hanaford's anecdote about rolling out vanilla Kubernetes and realizing the immense maintenance burden compared to a managed service like Amazon EKS perfectly illustrates the cost of reinvention when a "shiny new thing" distracts from efficiency.
• Burn Down Backlogs: Technical debt, particularly in security, can accumulate rapidly. This includes unpatched systems, outdated dependencies, and unaddressed vulnerabilities from bug bounty programs. Proactive management of these backlogs, even with small, consistent efforts, prevents them from becoming critical, all-hands-on-deck emergencies when a zero-day drops.

Focusing on People: Human-Centric Security Engineering

Given that 68% of breaches involve a human element and 96% of social engineering breaches stem from pretexting, phishing, and extortion, technical security must be designed with human behavior in mind:

• Design for Humans: Technical security controls should be intuitive and user-friendly. This means considering the workflow of developers, operations teams, and end-users. For example, a secure coding library that is easy to integrate and provides clear error messages will be adopted more readily than a technically superior but complex one.
• Build Paved Paths: This concept, popularized by Netflix, is highly technical. It involves creating default-secure configurations, automated deployment pipelines, and self-service security tools that make the secure option the easiest and most efficient. These "paved paths" are wide, allowing autonomy, but have "strong guardrails" (technical controls, policy enforcement) to prevent egregious errors. Examples include secure container images, pre-approved Infrastructure as Code (IaC) modules, or automated security testing integrated into CI/CD pipelines.
• Reduce Friction: Technical security should strive to make the secure way the easy way. While some friction is necessary (e.g., Multi-Factor Authentication (MFA)), excessive friction in daily workflows (e.g., overly complex access requests, slow security reviews) will lead users to find insecure workarounds. Technical solutions that automate security checks, provide immediate feedback, or integrate seamlessly into existing developer tools reduce this friction.

Embracing Collaboration: Integrated Security Operations

Technical security is not an isolated function; it thrives on collaboration:

• Be a Team Player & Lift All Boats: Technical security teams should actively engage with engineering, operations, and product teams. This means understanding their technical challenges, offering secure solutions that integrate into their workflows, and sharing security knowledge. For example, embedding security engineers within development teams or providing reusable security components.
• Be the Department of Yes: Instead of outright blocking technical initiatives, security teams should seek to enable them securely. This might involve finding alternative technical approaches, implementing compensating controls, or providing guidance on how to achieve a business goal while mitigating risks. When a "no" is necessary, it should be accompanied by a clear technical explanation and alternative solutions, fostering trust and understanding.

By adopting these principles, technical security teams can build more resilient, adaptable, and effective security programs that are deeply integrated into the organization's technical fabric, rather than being perceived as an external barrier.

Demo / Proof of Concept

▶ Watch: Strategies for 'being efficient': goal-oriented approach and avoiding reinven... (16:30)

Kevin Hanaford's presentation did not include a traditional technical demonstration or a live proof of concept in the sense of showing code or exploiting a vulnerability. Instead, the entire talk serves as a philosophical and strategic "proof of concept" for the imperfect security methodology itself. Hanaford's experience leading Trust Engineering at Discord, where he states that "engineering broadly speaking loves security," is presented as evidence of the practical success of these principles. He implies that by applying these concepts at Discord and in previous roles, he has fostered an environment where security is seen as an enabler rather than a blocker, leading to better outcomes and goodwill within the organization. The talk itself, therefore, is a demonstration of the ideas and their impact on organizational culture and security posture.

Defensive Implications

▶ Watch: Promoting 'embracing collaboration': being a team player and the 'department ... (28:30)

The "Imperfect Security" framework offers several critical defensive implications for security practitioners and leaders, shifting the focus from an unattainable ideal to pragmatic, impactful actions.

1. Strategic Risk Prioritization: Defenders must accept that they cannot secure everything perfectly. Instead, they should mind their business by understanding core business objectives and know their risks by identifying and prioritizing the most impactful threats to those objectives. This means focusing defensive efforts on assets and processes that, if compromised, would cause the most significant financial, reputational, or operational damage. This involves robust risk assessments and continuous monitoring, framing security investments in terms of business risk reduction rather than abstract security scores.
2. Proactive Planning for Imperfection: Security teams should plan for exceptions in their policies and controls. Instead of striving for bulletproof systems, anticipate legitimate deviations and build processes to manage, review, and monitor them. This reduces friction and prevents users from bypassing controls out of necessity. Furthermore, defenders must recognize that nothing they do will work forever; continuous iteration and adaptation of defensive strategies are essential.
3. Efficiency in Tooling and Processes: To combat the "infinite amount of work" with limited resources, defenders must be highly efficient. This means being goal-oriented rather than solution-oriented, defining desired security outcomes before selecting tools. Avoid reinvention by leveraging existing industry best practices, open-source solutions, or managed services instead of building custom tools for common problems. Actively burn down backlogs of technical debt, such as unpatched vulnerabilities or outdated configurations, to prevent them from becoming critical incidents.
4. Human-Centric Security Design: Given that 68% of breaches involve a human element and 96% of social engineering attacks are successful due to human manipulation, defensive strategies must prioritize the human factor. Design for humans by creating intuitive, low-friction security controls and processes. Build paved paths that make the secure option the easiest default for developers and employees, using automation and secure defaults. Reduce friction in security workflows, making the secure way the easy way, to encourage adoption and compliance rather than workarounds.
5. Fostering Collaboration and Trust: Defenders need to shed the "department of no" image. By actively being a team player and lifting all boats, security teams can build strong relationships with other departments. This involves understanding their needs, offering constructive solutions, and educating them on security in an approachable manner. Strive to be the department of yes, finding ways to enable business initiatives securely, rather than simply blocking them. When a "no" is necessary, it should be a rare, well-explained decision, backed by clear reasoning and potential alternatives. This builds goodwill, ensuring that when critical security issues arise, other teams are more likely to listen and collaborate.

By embracing these defensive implications, security teams can move beyond the Sisyphean task of achieving perfect security and instead build robust, adaptable, and business-aligned security programs that deliver tangible value and foster a culture of shared responsibility.

Key Takeaways

• Perfect Security is an Unattainable Ideal: The pursuit of perfect security is inefficient, costly, and yields diminishing returns. Security teams should abandon this goal and focus on pragmatic, impactful strategies.
• Operate Within Reality's Constraints: Acknowledge the infinite nature of security work, finite resources (budget, headcount), the unregulated nature of attackers, and the business's primary focus on revenue. Align security efforts with business priorities and manage risks effectively.
• Prioritize Efficiency and Outcomes: Be goal-oriented, defining desired security outcomes before selecting solutions. Avoid reinventing the wheel by leveraging existing tools and industry best practices, and proactively address technical debt to prevent future crises.
• Design Security for Humans: Recognize that the human element is central to most attacks. Build security controls and processes that are intuitive, low-friction, and make the secure option the easiest default, rather than relying solely on strict enforcement or complex procedures.
• Embrace Collaboration and Be an Enabler: Move beyond the "department of no" stereotype by fostering strong relationships across the organization. Be a team player, seek to enable business initiatives securely, and provide clear, constructive guidance, making "yes" the default response.
• Continuously Improve and Have Fun: Security is an ongoing journey of improvement. Hold yourself and your team accountable for continuous learning and adaptation. Injecting fun into the work environment can help manage the inherent stress of defending against threats.

About the Speaker(s)

Kevin Hanaford is the Trust Engineering Lead at Discord. His role involves overseeing a diverse engineering group that encompasses security, privacy, safety, and platform engineering. Outside of his professional life, Hanaford describes himself as "not a very technical person," primarily identifying as a musician and an "elder emo." He also enjoys skiing, spending time outdoors, camping, and lives in the Pacific Northwest, having previously been an "escapee from Alaska" who spent time in Seattle and California.