Human-Centered Security Meets AI: How to Navigate New Threats
Cornelia Puhze (switch)
Swiss Cyber Storm 2024 · Day 1 · Scenario
Overview
In an era increasingly dominated by sophisticated digital threats, Cornelia Puhze's talk, "Human-Centered Security Meets AI: How to Navigate New Threats," delivered at Swiss Cyber Storm, addresses a critical and often underestimated aspect of cybersecurity: the human element. Puhze, representing Switch, Switzerland's national research and education network, champions a paradigm shift from traditional, rule-based security awareness training to a more empathetic, human-centered approach. Her presentation dissects how the pervasive availability of Generative AI (GenAI) has dramatically altered the threat landscape, making highly targeted social engineering attacks cheaper, more scalable, and significantly harder for individuals to detect.

Key moments
- 0:00 Introduction to human-centered security
- 4:30 AI's impact: cheaper, targeted social engineering
- 10:00 The deepfake challenge: distinguishing reality from AI
- 16:00 Understanding the COM-B model for behavior change
- 20:00 The Elephant and Rider: targeting human instincts
- 26:00 "Slow down and frown": a meta-level defense
- 30:00 Q&A: Handling urgent internal communications
Human-Centered Security Meets AI: How to Navigate New Threats
Speakers: Cornelia Puhze (switch)
Conference: Swiss Cyber Storm
YouTube: https://www.youtube.com/watch?v=viGpnVb582Q
Overview
In an era increasingly dominated by sophisticated digital threats, Cornelia Puhze's talk, "Human-Centered Security Meets AI: How to Navigate New Threats," delivered at Swiss Cyber Storm, addresses a critical and often underestimated aspect of cybersecurity: the human element. Puhze, representing Switch, Switzerland's national research and education network, champions a paradigm shift from traditional, rule-based security awareness training to a more empathetic, human-centered approach. Her presentation dissects how the pervasive availability of Generative AI (GenAI) has dramatically altered the threat landscape, making highly targeted social engineering attacks cheaper, more scalable, and significantly harder for individuals to detect.
The core of Puhze's argument centers on the inadequacy of current security education methods in combating these evolving threats. She highlights that while technical solutions are vital, the majority of data breaches continue to involve human error, a statistic that fluctuates between 74% and 85% according to Verizon's data breach reports. This talk underscores the urgent need to re-evaluate how organizations prepare their employees to face threats like deepfakes and advanced phishing, advocating for strategies that appeal to human psychology and behavior rather than merely enforcing policies.
Puhze posits that the solution lies not in "shouting louder" or instilling fear, but in fostering trust, understanding human cognitive biases, and designing systems that make secure behavior intuitive. By integrating principles from behavioral economics and psychology, she proposes a "meta-level" approach to security awareness that equips individuals with the fundamental ability to slow down and critically evaluate suspicious situations, regardless of the attack vector. This forward-thinking perspective is crucial for any organization grappling with the intersection of human vulnerability and advanced AI-driven cyber threats.
Background
▶ Watch: Introduction to human-centered security (0:00)
The persistent challenge of securing the human element in cybersecurity is not new, but it has been profoundly exacerbated by the advent of widely accessible Generative AI (GenAI). For years, security practitioners have struggled with the reality that, despite significant investments in technology, humans remain the most exploited vulnerability. The Verizon Data Breach Investigations Report consistently highlights that a vast majority of breaches—ranging from 74% to 85%—involve a human component, whether through phishing, misconfiguration, or social engineering. This persistent statistic underscores a fundamental flaw in traditional cybersecurity strategies that often prioritize technical defenses over the complexities of human behavior.
Prior to GenAI, crafting highly targeted social engineering attacks, such as spear phishing or whale phishing, was resource-intensive, requiring significant time and effort to research targets and tailor convincing lures. However, GenAI has democratized this capability, making it "much cheaper to generate really targeted attacks" against individuals. This shift has fundamentally altered the threat landscape, as adversaries can now produce highly personalized and contextually relevant malicious content at an unprecedented scale, making it exceedingly difficult for even vigilant users to discern legitimate communications from sophisticated fakes.
The academic and research community has long recognized the limitations of traditional, rule-based security awareness training (SAT). As far back as 2015, research highlighted that security education often consisted of "repeating all policies and rules to everyone," akin to "shouting louder at someone who doesn't understand your language." This approach, Puhze argues, fails to meaningfully change behavior because it neglects the psychological underpinnings of human decision-making. Concepts from behavioral economics, such as Daniel Kahneman's System 1 and System 2 thinking, illustrate that humans primarily operate on intuitive, fast (System 1) thinking, reserving analytical, slow (System 2) thinking for complex tasks. Social engineering attacks masterfully exploit System 1, bypassing rational thought. Similarly, Robert Cialdini's principles of persuasion, often studied by sales professionals, are precisely what criminals leverage to manipulate individuals. Despite this understanding, practical security awareness programs have largely failed to integrate these behavioral insights, leading to a disconnect between theoretical knowledge and effective, real-world defensive behaviors. The problem, therefore, exists because security has too often treated humans as technical components that can be "patched" with information, rather than complex psychological entities requiring nuanced engagement.
Key Findings
▶ Watch: The deepfake challenge: distinguishing reality from AI (10:00)
Cornelia Puhze's talk unveils several critical findings that collectively advocate for a radical rethinking of cybersecurity education in the age of AI:
- AI-Driven Social Engineering Escalation: The most significant finding is that Generative AI (GenAI) has drastically reduced the cost and effort required to launch highly targeted social engineering attacks. This means adversaries can now produce convincing deepfakes, personalized phishing emails, and sophisticated scams at scale, making them "very hard or if not depending on what kind of attack to detect for people." This transforms social engineering from a niche, labor-intensive activity into a pervasive and easily accessible threat.
- Inability to Distinguish Reality from AI-Generated Content: Puhze presents alarming statistics on the human inability to differentiate between real and AI-generated content, especially images and videos (deepfakes). She cites research, already a year old at the time of the talk, indicating that "untrained observers have a 50% chance in like a minute to recognize whether something is real or not," and even more concerning, "only 60% of the times trained users with unlimited time were able to say this is actually a fake." This rapidly deteriorating capability renders traditional "look for the shadow" or "check the link" advice obsolete, highlighting that "we have no chance" if we rely solely on individual detection skills.
- Awareness vs. Capability Gap: While people are increasingly aware and concerned about AI-related cybercrime (as evidenced by reports from Cyfe/Cybersecurity Alliance and Proofpoint's Voice of the CISO report), there's a significant gap in their actual capability. CISOs believe their people "lack the skills and the knowledge and the tools required to defend their organization's data." This suggests that current awareness efforts are not translating into actionable defense mechanisms.
- Limitations of AI Fighting AI for the Human Element: Many CISOs express a desire to "use AI to protect against the people problem." Puhze partially agrees, acknowledging the need for "machines to fight machines" (e.g., deepfake detection tools). However, she strongly cautions against the "idea that you can just run an automated program and then after afterwards everyone knows about security," emphasizing that technology alone cannot solve the human element problem.
- Ineffectiveness of Traditional Rule-Based Training and Fear: Puhze critically assesses common mitigation practices, particularly traditional security awareness training (SAT) that focuses on rules. She argues this approach is like "shouting louder at someone who doesn't understand your language" and fails to "win the hearts and minds." Furthermore, she explicitly states that using fear as a messaging strategy is a "really bad idea" because it triggers an instinctive, irrational response (the "elephant") that hinders the analytical thinking required for security.
- The Power of Meta-Level Behavioral Training: The most promising finding is the efficacy of a "meta-level" approach to training. Instead of focusing on specific indicators (e.g., "read the link"), Puhze advocates teaching people to "slow down every time something comes in with urgency and requires action." She references research on mindfulness training in phishing, which, while not providing a specific percentage, implies a more robust defense than rule-based training (which showed "42% fewer clicks"). This approach targets the underlying cognitive biases exploited by social engineers, empowering individuals to question rather than react.
- Neglect of "Opportunity" and UX in Security: Puhze highlights that a crucial component of enabling secure behavior, the "opportunity" aspect from the COM-B model (Capability, Opportunity, Motivation -> Behavior), is largely neglected. This refers to the need for user-friendly, secure-by-design systems and processes (UX design) that make the desired secure action the easiest path. Without this, even motivated and capable individuals will struggle to maintain secure practices.
These findings collectively paint a picture of a security landscape where human psychology is the new battleground, and traditional technical or rule-based defenses are increasingly outmatched by AI-powered manipulation.
Technical Deep Dive
▶ Watch: Understanding the COM-B model for behavior change (16:00)
Cornelia Puhze's technical deep dive centers on the profound impact of Generative AI (GenAI) on social engineering and the psychological mechanisms that make humans vulnerable. She posits that GenAI has significantly lowered the barrier to entry for attackers, making targeted attacks cheaper and more widespread. This is achieved by automating the creation of highly convincing malicious content, such as personalized phishing emails, deepfake audio, and video. The challenge is amplified by the rapid evolution of this technology, where the quality of AI-generated content makes it nearly indistinguishable from reality, even for trained observers. Puhze cites data from a year prior to her talk, indicating that untrained individuals have only a 50% chance of detecting a deepfake within a minute, while trained users, given unlimited time, could only identify fakes 60% of the time. This highlights the inherent limitation of human perception against sophisticated AI.
To understand human vulnerability, Puhze introduces the concept of the "human operating system," drawing heavily on behavioral economics. She references Daniel Kahneman's work on System 1 and System 2 thinking.
- System 1 is our fast, automatic, intuitive, and emotional thinking, responsible for gut reactions and habitual responses. This system consumes less energy and is our default mode of operation.
- System 2 is our slow, analytical, rational, and effortful thinking, engaged for complex problem-solving.
Social engineers, Puhze explains, primarily target System 1, exploiting its quick responses and emotional triggers. She recommends Robert Cialdini's "Psychology of Persuasion," which outlines compliance principles such as urgency, authority, scarcity, and social proof—tactics routinely employed by criminals to manipulate individuals into desired actions.
Puhze then delves into models for behavior change, emphasizing their relevance to security:
- Linear Model: This simplified model suggests a progression: Awareness (recognizing security as an issue) -> Education and Training (acquiring knowledge and practical skills) -> Motivation (believing security is important and relevant) -> Behavior Change (making desired actions easy, repeatable, and habitual). She critiques current SAT programs for often stopping at awareness or knowledge, failing to teach actual skills (e.g., how to securely manage a password on a specific device) or foster genuine motivation.
- COM-B Model: A more scientific and comprehensive model, COM-B (Capability, Opportunity, Motivation -> Behavior) posits that behavior is a result of these three interacting components.
- Capability: The psychological and physical ability to perform the behavior (e.g., knowing what a strong password is, understanding how to use a password manager).
- Opportunity: The physical and social environment that enables the behavior (e.g., having a password manager provided, clear processes for reporting incidents, security by design in tools).
- Motivation: Reflective (conscious decision-making) and automatic (emotions, habits) processes that energize and direct behavior (e.g., understanding the purpose of security, caring about organizational data).
Puhze stresses that the "opportunity" aspect, particularly User Experience (UX) and security by design, is the most neglected area in human-centered security. She argues that security solutions should be designed to be inherently safe and easy to use, rather than imposing friction.
A key metaphor Puhze uses, drawn from Thaler and Sunstein's work, is the "elephant and the rider." The elephant represents our powerful, automatic, emotional, and instinctive System 1 responses, while the rider symbolizes our rational, analytical System 2. The elephant is much stronger and harder to control. Puhze contends that most security training exclusively targets the "rider," attempting to appeal to logic, while the "elephant" remains vulnerable to manipulation. When the elephant is attacked (e.g., through fear or urgency), it bolts, rendering the rider's rational commands useless. This explains why fear-based messaging is counterproductive in security; it triggers the elephant to flee or freeze, inhibiting rational analysis.
Puhze concludes her technical deep dive by advocating for a "meta-level" approach that addresses the elephant. Instead of teaching specific rules that are easily circumvented by evolving AI threats, she proposes teaching a universal principle: "slow down and frown." This concept, supported by research indicating that phishing is largely an "attention problem," encourages individuals to pause and question any message that evokes urgency, curiosity, or strong emotion, regardless of its channel (email, call, video). She references a study where mindfulness training yielded better outcomes than rule-based training (which still achieved "42% fewer clicks") by focusing on this meta-cognitive skill, and even a simple "frown" emoji indicator in emails showed considerable success in prompting caution. This strategy empowers individuals to defend against the underlying psychological manipulation, rather than chasing an ever-changing list of technical indicators.
Demo / Proof of Concept
▶ Watch: "Slow down and frown": a meta-level defense (26:00)
This talk did not feature a live technical demonstration or proof of concept in the traditional sense, such as showcasing a tool or a specific attack vector. Instead, Cornelia Puhze effectively utilized interactive audience engagement and a compelling anecdote to illustrate her core concepts and the practical implications of human-centered security.
Early in her presentation, Puhze engaged the audience with a series of "yes or no" questions, subtly demonstrating how easily individuals can be guided or "manipulated" into agreement, reflecting the psychological principles exploited in social engineering. This served as a soft "proof of concept" for the audience's own susceptibility.
The primary illustrative example Puhze provided was the story of "Julia," a successful professional who confidently believed she was immune to phishing attacks based on her "gut feeling." Through her conversation with Julia, Puhze revealed several critical failures of traditional security awareness:
- Reputational Damage: Julia perceived the IT security team as "a bunch of weirdos" who were "trying to trick" her with phishing tests, highlighting a severe lack of trust and alliance between security professionals and end-users.
- Lack of Foundational Skills: Julia admitted she didn't know how to "read a link" (understand a URL) or identify malicious file extensions, despite her confidence. This exposed the gap between perceived awareness and actual capability.
- Reliance on Unreliable Intuition: Julia's sole defense mechanism was her "gut feeling," which Puhze identified as a dangerous reliance on System 1 thinking, precisely what sophisticated social engineering targets.
The "Julia" anecdote served as a powerful, real-world "demonstration" of why traditional, rule-based security training fails, why trust is crucial, and why a meta-level approach is necessary. By using this relatable narrative, Puhze effectively conveyed the practical challenges and validated her argument for shifting towards psychological and behavioral interventions rather than technical or policy-driven ones.
Defensive Implications
▶ Watch: Q&A: Handling urgent internal communications (30:00)
The insights from Cornelia Puhze's talk provide crucial defensive implications for organizations grappling with AI-driven threats and the persistent human element problem:
- Shift from Rule-Based to Meta-Level Training: Defenders must abandon the outdated model of teaching specific, constantly changing technical rules (e.g., "check the link," "watch for shadows in deepfakes"). Instead, the focus should be on meta-level behavioral training. The core principle is to teach individuals to "slow down and frown" whenever they encounter messages that evoke urgency, curiosity, or strong emotions, regardless of the channel (email, text, video call). This cultivates a critical mindset that is more resilient to evolving attack vectors.
- Foster Trust and Psychological Safety: Security teams must actively cultivate an environment of trust with end-users. Puhze's "Julia" anecdote vividly illustrates the counterproductive effect of being perceived as "weirdos" or adversaries. Defenders need to be seen as allies, making it safe and easy for employees to report suspicious activities without fear of blame or ridicule. This encourages vital threat intelligence sharing from the front lines.
- Prioritize User Experience (UX) and Security by Design: The "opportunity" aspect of the COM-B model is paramount. Security solutions and processes should be designed with the user in mind, making the secure choice the easiest and most intuitive one. This means investing in UX design for security tools, streamlining reporting mechanisms, and implementing security by design principles in all applications to reduce friction and human error.
- Leverage AI to Protect, Not Just Detect: While automated programs alone cannot solve the "people problem," organizations should strategically deploy AI-powered tools to create a safer digital environment for employees. This includes advanced deepfake detection systems, sophisticated email filters, and behavioral analytics that can flag anomalous activity before it reaches the human decision-maker. The goal is to make it as "safe as possible for them to actually be online."
- Educate on Persuasion Principles (Training the "Elephant"): Instead of solely appealing to the rational "rider," security education must acknowledge and train the emotional "elephant." This involves teaching employees about the compliance principles (e.g., urgency, authority, scarcity, social proof) that social engineers exploit. Understanding how they are being manipulated can empower individuals to recognize and resist these tactics.
- Abandon Fear-Based Messaging: Using fear as a primary communication strategy is counterproductive. It triggers instinctive System 1 responses that inhibit rational thought and can lead to anxiety, avoidance, and disengagement. Defenders should instead focus on positive reinforcement, empowering language, and connecting security to an individual's higher purpose or the organization's mission.
- Creative and Multi-Channel Communication for Urgent Matters: For legitimate urgent requests, organizations must move beyond single-channel, high-urgency emails that mimic phishing attempts. As highlighted in the Q&A, this requires creative solutions:
- Multi-channel verification: Sending critical alerts via two different, independent channels (e.g., email followed by an internal chat message or a direct communication from a team lead).
- Involving team leads: Empowering managers to communicate urgent actions, adding a layer of trusted authority.
- Process re-evaluation: Questioning whether certain "urgent" processes are truly necessary or if they can be redesigned to be less susceptible to mimicry by attackers.
By implementing these defensive implications, organizations can move towards a more resilient security posture that acknowledges human nature, leverages technology intelligently, and fosters a culture of security rather than one of compliance and fear.
Key Takeaways
- AI Transforms Social Engineering: Generative AI has made highly targeted phishing and deepfake attacks cheaper, more scalable, and significantly harder for individuals to detect, rendering traditional rule-based security awareness training largely ineffective.
- Prioritize Meta-Level Behavioral Training: Instead of teaching specific technical indicators, empower users with a universal "slow down and frown" approach to critically evaluate messages that trigger urgency, curiosity, or strong emotions, regardless of the communication channel.
- Build Trust and Alliance with Security Teams: Organizations must foster an environment where employees see security professionals as allies, not adversaries. Trust is crucial for encouraging reporting of suspicious activities and for the overall effectiveness of security programs.
- Integrate UX and Security by Design: Secure behavior should be made easy and intuitive through user-friendly systems and processes. Neglecting the "opportunity" aspect (from the COM-B model) by not designing for usability undermines even well-intentioned security efforts.
- Train the "Elephant" (Address System 1 Thinking): Security education must go beyond rational appeals to the "rider" and address the emotional, instinctive "elephant" (System 1). Teach employees about the psychological persuasion principles (e.g., urgency, authority) that social engineers exploit.
- Avoid Fear-Based Messaging: Instilling fear is counterproductive; it triggers emotional responses that inhibit rational decision-making and leads to avoidance rather than engagement with security practices.
About the Speaker(s)
Cornelia Puhze is a prominent advocate for human-centered security, working at Switch, Switzerland's national research and education network. Her role involves a crucial job of increasing awareness and "bringing the human into the loop" in cybersecurity. Puhze is part of Switch's security awareness team, now evolving into the human-centered security team.
Her work encompasses a wide range of activities:
- Security Advocacy: Engaging with both technical audiences and end-users to explain the importance of human-centered security.
- Events and Training: Organizing and conducting training for both cybersecurity practitioners and general end-users.
- Public Speaking: Regularly presenting at conferences like Swiss Cyber Storm and acting as an expert voice in media outlets.
- Innovative Awareness Programs: Developing and implementing "security awareness adventures," which include serious games like security-themed escape rooms and role-playing games akin to Dungeons and Dragons, translated to address social engineering scenarios.
Puhze is dedicated to fostering a deeper understanding of human behavior in cybersecurity and is actively involved in developing educational initiatives, including a new human-centered security course co-developed with Teta.
Reviews
Dr. Zero (Offensive Security Researcher) — WEAK
We are still doing awareness talks that discover Daniel Kahneman a decade late and slap GenAI on the title to get past the CFP board. Puhze correctly diagnoses that rule-based phishing training is hot garbage, but replacing it with mindfulness is not a structural defense.
Heather Calloway (CISO) — SOLID
A behavioral critique of why traditional security awareness training fails, made more urgent by GenAI's ability to scale targeted attacks. The core argument against rule-based training is sound, but the real takeaway for leadership is about funding security UX rather than just teaching employees mindfulness.