Nerds on a plane: what we can learn from the aviation industry

Nerds on a plane: what we can learn from the aviation industry

Speakers: Sarah Young, Principal Security Advocate, Microsoft

Conference: CrikeyCon

YouTube: https://www.youtube.com/watch?v=BCURb4e5K9A

Overview

In her compelling CrikeyCon talk, "Nerds on a plane: what we can learn from the aviation industry," Sarah Young, Principal Security Advocate at Microsoft, delves into the profound lessons cybersecurity professionals can glean from the highly mature and safety-critical aviation sector. Young challenges the common perception that security challenges are unique, asserting that the fundamental principles of risk management and incident response are universal. By dissecting historical aircraft accidents and the robust safety frameworks developed in their wake, she illustrates how aviation's systematic approach to mitigating risk, fostering a positive safety culture, and conducting transparent post-mortems offers an invaluable blueprint for improving cybersecurity practices.

The talk provides a unique perspective, moving beyond purely technical solutions to emphasize the critical interplay of human factors, process, and culture in preventing catastrophic failures. Young leverages her extensive experience in both security and frequent air travel to bridge the gap between two seemingly disparate fields, demonstrating that the "Swiss cheese model" of safety and comprehensive incident investigations are not just for engineers and pilots, but vital tools for anyone striving to build more resilient and secure systems. Her analysis underscores that while the consequences of cyber incidents may differ from plane crashes, the underlying mechanisms of failure and the strategies for learning from them bear striking similarities.

This article will explore Young's arguments, detailing the aviation safety models, specific accident case studies, and their direct implications for cybersecurity. It aims to provide a comprehensive understanding of how adopting a more aviation-like mindset—one that prioritizes transparency, learning from mistakes, and balancing risk with operational necessity—can significantly enhance the effectiveness of security programs across all industries.

Background

▶ Watch: Connecting aviation safety lessons to security (2:00)

The foundation of Sarah Young’s argument rests on the premise that managing risk is not a problem exclusive to cybersecurity but a universal challenge faced by all safety-critical industries. Aviation, having evolved over a century into one of the safest modes of transport, provides a rich historical context for effective risk management.

A cornerstone of aviation safety is the Swiss cheese model of safety, originally proposed by James Reason. This model posits that hazards are prevented from leading to accidents by multiple layers of defensive controls, much like slices of Swiss cheese. Each slice has "holes"—latent failures or weaknesses—but for an accident to occur, all the holes in the various layers must momentarily align, allowing the hazard to pass unimpeded to the eventual accident. These layers can include technical safeguards, procedural rules, human training, and organizational culture. Young highlights how this model directly parallels security practices, where multiple controls (firewalls, MFA, training) are deployed to prevent breaches.

Aviation industries also operate under rigorous Safety Management Systems (SMS). These systems are comprehensive frameworks designed to systematically manage safety risks. Young outlines key components of an SMS:

• Safety Policy and Objectives: Clear statements of an organization's commitment to safety and measurable goals.
• Safety Risk Management: A structured process for identifying hazards, assessing their risks (severity and probability), and developing mitigation strategies. This often involves determining whether risks are tolerable or require further control implementation.
• Safety Assurance: Continuous monitoring and evaluation of the effectiveness of implemented controls to ensure they are performing as intended. This includes audits, reviews, and performance measurement.
• Safety Promotion: Activities aimed at fostering a positive safety culture, including training, education, and communication to ensure all personnel understand their safety responsibilities.

Young draws a direct comparison to cybersecurity, noting that these SMS components are strikingly similar to what security teams attempt to do with risk assessments, security controls, and awareness training. However, she argues that aviation often executes these with greater depth and a more mature safety culture. A critical aspect of this is a just culture, where individuals are encouraged to report honest mistakes without fear of undue punishment, fostering learning rather than blame. This contrasts sharply with some cybersecurity environments where finger-pointing after an incident can stifle reporting and prevent genuine learning. Aviation also emphasizes a learning culture, where lessons from incidents are thoroughly analyzed and shared to prevent recurrence.

Crucially, aviation has adopted a philosophy of balancing production and protection. The industry acknowledges that eliminating all risk would mean no planes ever take off, rendering the business non-existent. Instead, they accept calculated risks while implementing robust controls. This pragmatic approach is a lesson for security professionals who often default to "no" when faced with risk, potentially alienating business units and leading them to bypass security measures. The historical data from aviation, showing a significant decline in accidents over decades due to rigorous post-mortems and continuous improvement, serves as a powerful testament to the effectiveness of these frameworks. Young notes that while cyber incidents rarely involve the immediate loss of life as dramatically as plane crashes, the principle of learning from every failure is equally vital.

The process for assessing aircraft accidents further highlights aviation's maturity. Investigations are incredibly thorough, involving:

1. Evidence Collection: Gathering physical wreckage, black box data (voice and flight recorders), and other relevant information.
2. Examination: Detailed analysis of collected evidence, often reconstructing parts of the aircraft in large warehouses.
3. Expert Collaboration: Bringing together diverse experts (investigators, metallurgists, aerodynamicists) to understand every facet of the failure.
4. Draft Report & Internal Review: An initial report is compiled and reviewed by internal experts.
5. External Peer Review: The draft is then reviewed by external, independent peers.
6. Approval & Public Release: The final report is approved and made publicly available.

This transparency and public sharing of detailed findings are crucial for industry-wide learning and improvement, a stark contrast to the often secretive nature of cyber incident reporting, though mandatory reporting regulations are beginning to shift this paradigm.

Key Findings

▶ Watch: Aviation's safety management systems and culture (6:30)

Sarah Young's talk distills several critical findings from aviation safety that have profound implications for cybersecurity:

• Risk Management is a Universal Discipline: The fundamental challenges of identifying, assessing, mitigating, and accepting risk are not unique to cybersecurity. Other industries, particularly aviation, have mature models and processes that security professionals can adapt. This broadens the perspective beyond technical solutions to include organizational, human, and procedural elements.
• Structured Safety Management Systems Drive Improvement: Aviation's Safety Management Systems (SMS) provide a robust, systematic framework for managing risk. Their emphasis on clear policies, continuous monitoring (safety assurance), and proactive education (safety promotion) offers a template for building more resilient security programs that move beyond ad-hoc responses.
• Culture is Paramount for Safety: A positive safety culture, characterized by a just culture (where honest mistakes can be reported without fear of retribution) and a learning culture (where lessons from incidents are shared widely), is more effective than a blame-oriented environment. This directly impacts the willingness of employees to report security anomalies or admit errors, which is crucial for early detection and rapid response.
• Transparency and Thorough Post-Mortems are Essential: Aviation's commitment to exhaustive, multi-expert, and public incident investigations (post-mortems) is a key driver of continuous improvement. The detailed analysis of why failures occurred, rather than who was at fault, allows for systemic changes that prevent recurrence. This contrasts with the often-secretive and blame-focused nature of many cybersecurity incident responses.
• Balance Between Production and Protection is Non-Negotiable: Aviation demonstrates that absolute risk elimination is impossible and impractical. Instead, a pragmatic approach that balances operational necessity ("production") with safety ("protection") is required. Security must learn to enable the business securely, rather than simply saying "no," to avoid being bypassed.
• Automation Can Lead to Skill Atrophy: While automation enhances efficiency, over-reliance can degrade essential human skills required for manual intervention and problem-solving when automated systems fail. This is particularly relevant in the age of AI, where fundamental understanding of underlying processes remains critical.
• Human Fallibility is a Constant Factor: Even highly trained and experienced individuals make mistakes. Systems and procedures must be designed with this in mind, incorporating checks and balances, and fostering environments where junior staff feel empowered to speak up.
• Procedures Reduce Errors and Misunderstandings: Well-defined and consistently enforced procedures are vital for scaling operations, ensuring consistency, and reducing the likelihood of human error or miscommunication, particularly in high-stress situations.
• Everyone is Susceptible to Social Engineering: The talk highlights that even security professionals can fall victim to sophisticated social engineering tactics. Continuous awareness, skepticism, and a willingness to question assumptions are necessary for all.
• "You Don't Know What You Don't Know": New technologies introduce unforeseen risks and vulnerabilities. Acknowledging this inherent uncertainty requires a proactive, investigative mindset to uncover and address novel threats before they lead to catastrophic failures.

Technical Deep Dive

▶ Watch: How aviation thoroughly assesses accidents for learning (11:50)

Sarah Young's technical deep dive into aviation safety principles is primarily achieved through a series of compelling accident case studies, each illustrating specific layers of the Swiss cheese model that failed and offering direct parallels to cybersecurity challenges. These incidents underscore the complex interplay of technical, human, and procedural factors in critical systems.

Air France Flight 447 (2009)

This accident involved an Airbus A330 en route from Rio de Janeiro to Paris. The core technical issue was the temporary blockage of the aircraft's pitot tubes by ice. Pitot tubes are crucial sensors that measure airspeed, a fundamental parameter for flight control. When these tubes became blocked, the aircraft's autopilot disengaged, handing control back to the human pilots.

The critical failure here was the pilots' response. The two junior pilots on duty panicked. Instead of holding the aircraft steady, as would typically be required, they made incorrect control inputs, inadvertently causing an aerodynamic stall. An aerodynamic stall occurs when the wings lose lift because air is no longer flowing fast enough over their surface. Despite the captain returning to the cockpit and identifying the problem, the aircraft was too low to recover, leading to the loss of all 228 lives.

Security Parallel: This highlights the danger of over-reliance on automation and the atrophy of essential manual skills. In cybersecurity, this translates to relying solely on automated security tools (e.g., SIEM, EDR) without maintaining the fundamental skills for manual threat hunting, incident analysis, or system hardening when those tools fail, are misconfigured, or generate false positives. If an automated defense system disengages or provides incorrect information, human responders must possess the expertise to take over effectively.

De Havilland Comet Crashes (1950s)

The De Havilland Comet was a revolutionary aircraft, the world's first commercial jetliner. However, it suffered a series of catastrophic in-flight breakups early in its service life. Post-accident investigation, conducted without the benefit of black boxes, revealed a novel technical flaw: the square-shaped passenger windows with rounded corners were experiencing metal fatigue under the stress of repeated cabin pressurization cycles. Jet planes fly at much higher altitudes (above 10,000 feet) where the atmospheric pressure is too low for human respiration, necessitating a pressurized cabin. The corners of the square windows were points of high stress concentration, leading to cracks that propagated and eventually caused explosive decompression and structural failure.

Security Parallel: This is a classic example of "you don't know what you don't know" when introducing new technology. The engineers of the time understood pressurization but did not fully comprehend the long-term material fatigue implications of specific structural designs under cyclic stress. In cybersecurity, this mirrors the introduction of novel technologies (e.g., new cloud architectures, AI/ML systems, quantum computing) that may harbor unknown vulnerabilities or introduce unforeseen attack vectors due to their complexity or unique interactions within existing systems. Proactive security research and rigorous testing for unknown unknowns become critical. This led to the industry standard of oval-shaped windows, distributing pressure more evenly.

Airblue Flight 202 (2010)

This Pakistani airline flight crashed into a mountain range near Islamabad. The investigation, primarily relying on the cockpit voice recorder, uncovered a deeply problematic crew resource management and safety culture issue. The captain had subjected the first officer to intense, unwarranted grilling throughout the flight, creating a hostile environment. As they approached Islamabad, the first officer noticed they were veering off course but, intimidated by the captain, hesitated to assertively challenge his decisions. The captain ignored the first officer's tentative warning ("Sir, I think we're in the wrong place"). Compounding this, the airport lacked radar for ground control, meaning air traffic controllers were entirely reliant on pilot reports. The captain also ignored the minimum descent level of 2500 feet for that airport, continuing to descend into dangerous terrain.

Security Parallel: This illustrates how toxic culture and hierarchical intimidation can directly lead to catastrophic failure. If junior security analysts or developers fear retribution for reporting anomalies, questioning senior decisions, or admitting mistakes, critical issues can go unaddressed. A lack of monitoring and visibility (no radar) meant external checks were absent. Ignoring established security baselines or procedures (minimum descent level) is a direct risk. This incident underscores the need for a just culture where even junior team members feel empowered to speak up about potential risks without fear.

Tenerife Air Disaster (1977)

The single deadliest accident in aviation history involved two Boeing 747 jumbo jets, a KLM flight and a Pan Am flight, colliding on the runway at Tenerife Airport in the Canary Islands. This disaster was a confluence of numerous, seemingly minor failures aligning—a perfect storm of Swiss cheese holes.

• External Event: Both planes were diverted to the small Tenerife airport due to a bomb threat at a larger airport, leading to congestion.
• Environmental Factors: Dense fog rapidly rolled in, reducing visibility to near zero.
• Infrastructure Deficiencies: Tenerife airport lacked ground radar, preventing air traffic controllers from knowing the exact position of aircraft on the runway in the fog.
• Human Factors & Communication: The KLM captain, eager to depart due to flight duty time limits, decided to refuel, making his aircraft heavier. Crucially, there were non-standard radio communications between the KLM cockpit and air traffic control. The KLM captain misheard or misinterpreted instructions, believing he had clearance for takeoff when he only had clearance to proceed to the runway. The Pan Am crew also used non-standard phraseology ("Yes, okay") which added to the confusion. The KLM first officer expressed concern, asking, "Is Pan Am not clear of the runway?" but was dismissed by the captain.

Security Parallel: This is a powerful illustration of how multiple, seemingly unrelated security "holes" can align to create a catastrophic event. An external threat (bomb threat) led to an overloaded, unprepared environment. Lack of visibility (fog, no radar) hindered situational awareness. Communication breakdowns, human pressure (captain's eagerness), and the disregard of a junior's warning all contributed. This emphasizes the need for layered defenses, robust communication protocols, and mechanisms to escalate concerns regardless of seniority.

Air India Flight 182 (1985)

This flight was destroyed over the Atlantic by a bomb placed in checked luggage by a Sikh terrorist group. This incident highlighted critical failures in security procedures and human vulnerability to social engineering.

• Social Engineering: The terrorist successfully pressured a check-in agent in Vancouver to check his bags all the way through to India, despite policy requiring re-checking at an intermediate stop. He used tactics like feigned outrage and claiming business class status.
• Procedural Failure & Training Gaps: At Toronto, where the bags were transferred, the luggage X-ray machine was broken. Baggage handlers were using handheld scanners, but their training was inadequate. One handler noted a "soft beep" from the bomb-laden bag but had been taught to only act on a "loud beep," so they let it through.
• Policy Weakness: At the time, policy allowed bags to be checked into the hold without the accompanying passenger being on the flight. This meant the terrorist could plant the bomb without intending to die with it.

Security Parallel: This is a stark lesson in social engineering susceptibility and the importance of strict adherence to security policies. The check-in agent, under pressure, deviated from protocol. Broken security tools (X-ray) combined with inadequate training for backup procedures (handheld scanners) created a critical vulnerability. The policy allowing unaccompanied baggage was a fundamental design flaw. This event led to the ubiquitous practice of offloading bags if a passenger fails to board, a direct security learning from this tragedy. It reminds security professionals that tools are only as effective as the processes and training supporting them, and that every human is a potential target for social engineering.

Demo / Proof of Concept

▶ Watch: Tenerife disaster: multiple layers of human failure (20:50)

The talk "Nerds on a plane: what we can learn from the aviation industry" is a conceptual and analytical presentation, drawing parallels between two distinct fields. It does not include a live demonstration or a proof of concept of any specific technical tool or exploit. Instead, the "proof" of its concepts lies in the historical data and well-documented accident investigations from the aviation industry itself.

Defensive Implications

▶ Watch: Crucial lessons from aviation for tech and security (26:50)

The lessons from aviation safety offer a transformative framework for cybersecurity defenders, moving beyond purely technical solutions to encompass holistic risk management.

1. Embrace the Swiss Cheese Model for Layered Defense: Security teams must design and implement layered security controls across people, process, and technology. No single control is perfect; each has "holes." The goal is to ensure that for a threat to succeed, multiple independent controls must fail simultaneously. This means investing in diverse defenses, from network segmentation and endpoint protection to robust identity management and comprehensive employee training.
2. Adopt Aviation's Safety Management System (SMS) Principles: Structure security programs with clear security policies and objectives, a systematic security risk management process (identifying threats, assessing vulnerabilities, calculating impact/likelihood), continuous security assurance (monitoring control effectiveness, regular audits, penetration testing), and proactive security promotion (ongoing security awareness, training, and fostering a security-conscious culture).
3. Cultivate a Just and Learning Security Culture: Encourage employees, from junior analysts to senior executives, to report security incidents, near-misses, and even honest mistakes without fear of blame. Implement mechanisms for anonymous reporting where appropriate. Prioritize learning from every incident over assigning individual fault, and actively share these lessons across the organization to prevent recurrence.
4. Prioritize Thorough, Blame-Free Post-Mortems: After any security incident, conduct exhaustive post-mortems that investigate not just the technical chain of events, but also the human factors, procedural breakdowns, and cultural issues that contributed. Avoid finger-pointing; instead, focus on identifying systemic weaknesses. Document findings meticulously and share them internally, and externally where appropriate and legally permissible, to contribute to collective industry knowledge.
5. Balance Security with Business Enablement: Shift from a default "no" to a "how can we do this securely?" mindset. Understand business objectives and work collaboratively to find secure ways to achieve them, rather than creating roadblocks. This pragmatic approach ensures security is seen as an enabler, not an impediment, reducing the likelihood of business units bypassing security for expediency.
6. Maintain Core Manual Skills Alongside Automation: While security automation (e.g., SOAR, AI-driven detection) is crucial for scale and speed, ensure that security professionals retain and regularly practice fundamental skills for manual analysis, incident response, and system configuration. When automated systems fail or encounter novel threats, human expertise will be indispensable for problem-solving and recovery.
7. Implement and Enforce Robust Security Procedures: Develop clear, well-documented, and consistently enforced procedures for all critical security operations, from access management and change control to incident response and patch management. Procedures reduce the likelihood of human error, ensure consistency, and provide a reliable framework, especially during high-stress situations.
8. Conduct Continuous and Realistic Social Engineering Awareness: Acknowledge that everyone, including security professionals, is susceptible to social engineering. Implement ongoing, varied, and realistic phishing simulations, social engineering tests, and training programs. Emphasize critical thinking, verification processes, and the importance of questioning unusual requests.
9. Proactively Address "Unknown Unknowns" in New Technologies: When adopting new technologies (e.g., AI, IoT, quantum computing), assume there are unforeseen risks and vulnerabilities. Dedicate resources to proactive security research, threat modeling, and novel attack surface analysis. Build security architectures that are adaptable and resilient to emerging threats that are not yet fully understood.
10. Advocate for Transparency and Information Sharing: Support initiatives for mandatory breach reporting and industry-wide threat intelligence sharing. The collective benefit of understanding adversaries' tactics, techniques, and procedures (TTPs) and learning from others' failures far outweighs the perceived risks of disclosure.

Key Takeaways

• Risk management is a universal discipline: Cybersecurity can significantly improve by adopting the mature, systematic approaches to risk assessment and mitigation developed in safety-critical industries like aviation.
• Culture dictates safety outcomes: Fostering a just culture (where honest mistakes can be reported without fear) and a learning culture (where incident lessons are widely shared) is paramount for effective security, encouraging transparency and continuous improvement.
• Thorough post-mortems are non-negotiable: Detailed, blame-free investigations into security incidents, focusing on systemic failures rather than individual fault, are essential for identifying root causes and implementing changes that prevent recurrence.
• Automation requires human oversight and skill maintenance: While automation enhances efficiency, over-reliance can lead to the atrophy of critical manual skills. Security teams must retain and practice fundamental expertise for when automated systems fail or encounter novel threats.
• Balance protection with production: Effective security enables business operations securely, rather than simply blocking them. This requires understanding business needs and finding pragmatic solutions that manage risk rather than attempting impossible eradication.
• Everyone is a target for social engineering: Continuous, realistic training and a culture of skepticism are vital, as even security experts are susceptible to sophisticated social engineering tactics.

About the Speaker(s)

Sarah Young is a Principal Security Advocate at Microsoft, a role that primarily involves engaging with people, speaking at conferences, and contributing to Microsoft's security content through videos and other media. With a career spanning many years in the security industry, she brings a wealth of experience to her discussions. Young is a self-confessed frequent flyer, traveling extensively for her work, which ironically sparked her deep interest in aviation safety after initially being scared of planes. This personal motivation led her to understand the intricacies of aircraft operation and safety protocols. While she clarifies she is not a "hardcore aviation geek," she deeply appreciates the engineering and robust systems inherent in the industry. Her background also includes working for a "big four" firm, where she gained significant experience in conducting risk assessments.