Getting Catfished by AI: Why Your CEO Might Not Be Who You Think They Are

James McQuiggan

CypherCon 9.0 (2026)

Overview

In an era where digital identities are increasingly malleable, James McQuiggan's CypherCon talk, "Getting Catfished by AI: Why Your CEO Might Not Be Who You Think They Are," delivers a stark warning about the escalating threat of AI-powered deepfakes and sophisticated social engineering. McQuiggan, the founder of Apparent Security LLC, demonstrates firsthand how easily convincing deepfake videos can be created, setting the stage for a deep dive into the technical underpinnings, real-world impacts, and crucial defensive strategies against these evolving threats. The presentation opens with a compelling deepfake of CypherCon's keynote speaker, Michelle, immediately illustrating the talk's central premise: the question is no longer if a digital representation is a deepfake, but how easily and quickly it was made.

Watch on YouTube

Visual summary for Getting Catfished by AI: Why Your CEO Might Not Be Who You Think They Are by James McQuiggan
Visual summary for Getting Catfished by AI: Why Your CEO Might Not Be Who You Think They Are by James McQuiggan

Key moments

  1. 2:00 Speaker's deepfake introduction and challenge
  2. 4:30 $25 million deepfake scam in Hong Kong
  3. 7:30 Ferrari audio deepfake & effective verification questions
  4. 25:30 Emerging digital arrest scams via deepfake video
  5. 28:50 Detecting a deepfake during a job interview
  6. 46:50 Shift from spotting deepfakes to verification
  7. 54:50 Key actions: Audit AI, executive digital footprint, tabletop exercises

Getting Catfished by AI: Why Your CEO Might Not Be Who You Think They Are

Speakers: James McQuiggan, Founder, Apparent Security LLC

Conference: CypherCon

YouTube: https://www.youtube.com/watch?v=MO-sophc5C0

Overview

In an era where digital identities are increasingly malleable, James McQuiggan's CypherCon talk, "Getting Catfished by AI: Why Your CEO Might Not Be Who You Think They Are," delivers a stark warning about the escalating threat of AI-powered deepfakes and sophisticated social engineering. McQuiggan, the founder of Apparent Security LLC, demonstrates firsthand how easily convincing deepfake videos can be created, setting the stage for a deep dive into the technical underpinnings, real-world impacts, and crucial defensive strategies against these evolving threats. The presentation opens with a compelling deepfake of CypherCon's keynote speaker, Michelle, immediately illustrating the talk's central premise: the question is no longer if a digital representation is a deepfake, but how easily and quickly it was made.

McQuiggan's talk transcends mere technical exposition, weaving in practical examples, alarming statistics, and a call to action for organizations and individuals alike. He highlights how cybercriminals are leveraging advancements in AI to amplify deception, exploit human trust, and bypass traditional security measures, with potentially catastrophic financial and reputational consequences. From CEO impersonation scams demanding multi-million dollar wire transfers to the emerging threat of "digital arrests," the landscape of social engineering is being radically reshaped by generative AI and agentic AI.

The core message of the presentation is that while AI presents unprecedented challenges, the fundamental principles of cybersecurity remain relevant, albeit with a renewed emphasis on human resilience and robust verification processes. McQuiggan advocates for a strategic shift from attempting to "spot" increasingly sophisticated deepfakes to implementing a culture of "trust and verify." This article will explore the intricate details of these attacks, the tools and techniques used, and actionable insights for strengthening defenses in this rapidly evolving threat environment.

Background

▶ Watch: Speaker's deepfake introduction and challenge (2:00)

The proliferation of sophisticated AI technologies has ushered in a new era of cyber threats, fundamentally altering the landscape of social engineering and impersonation. Historically, scams like the "CEO fraud" or "business email compromise" relied on convincing text or voice impersonations, often with noticeable flaws. However, the advent of generative AI and deepfake technologies has blurred the lines between reality and fabrication, making these attacks significantly more potent and difficult to detect.

McQuiggan illustrates this evolution by recalling the familiar "gift card scam," where employees would receive text messages from their "boss" requesting gift card purchases. While these scams were often rudimentary, they exploited a basic human tendency to comply with authority. Today, the same underlying social engineering principles are being applied with unprecedented sophistication, leveraging AI-generated audio and video to create highly believable impersonations of senior executives. The problem is not new, but the tools available to adversaries have become exponentially more powerful.

Several high-profile incidents underscore the gravity of this problem. A notable case involved an employee in a Hong Kong finance department who was socially engineered into transferring $25 million. This elaborate scheme involved chat messages, emails, and eventually deepfake video calls impersonating not only the CFO and CEO but also board members. The attack exploited cultural norms of hierarchy and respect, demonstrating how AI can be integrated into multi-channel campaigns to build rapport and coerce victims. Similarly, Ferrari was targeted by an audio deepfake of its CEO, attempting to induce a wire transfer from the CFO. In this instance, the CFO's quick thinking—asking a specific, personal question about a lent book—exposed the scammer's inability to respond, leading them to hang up. These incidents highlight the severe financial implications and the critical role of human verification.

The financial impact of AI-powered attacks is staggering. McQuiggan cites a Dark Reading report indicating that 85% of organizations have been hit by some type of AI-powered attack, with 89% dealing with adversary attacks. The average cost of an AI breach is estimated at $4.6 million, a figure McQuiggan believes is likely conservative and will rise significantly. Projections indicate a 1200% increase in AI fraud by 2025, signaling an exponential growth in these types of incidents. This surge is directly linked to the widespread adoption of AI tools, both legitimate and malicious. While many organizations are grappling with the opportunities presented by AI, cybercriminals are rapidly integrating these technologies to amplify deception, create polymorphic malware, and launch more targeted and effective attacks, exploiting the inherent human tendency to trust. The emerging threat of agentic AI, where AI systems act as autonomous "employees," further complicates the security landscape, raising questions about how to secure and defend against AI-driven entities within an organization.

Key Findings

▶ Watch: Ferrari audio deepfake & effective verification questions (7:30)

James McQuiggan's presentation reveals several critical insights into the current state of AI deepfakes and their impact on cybersecurity:

  • Ease and Speed of Deepfake Creation: Deepfakes are no longer the exclusive domain of state-sponsored actors or highly skilled technicians. McQuiggan demonstrates that compelling deepfake videos, complete with accurate lip-syncing, can be created in under four minutes using readily available tools and APIs. His custom application, "Fakemaker," reduced the process from over 10 minutes to less than four, underscoring the accessibility and efficiency of these malicious capabilities.
  • AI Amplifies Social Engineering Effectiveness: AI, particularly Large Language Models (LLMs), significantly enhances traditional social engineering tactics. AI-powered phishing emails are 14 times more likely to succeed, meaning approximately one in seven employees could fall victim to a well-crafted AI-generated phish. This amplification extends to voice and video deepfakes, making CEO impersonation scams more convincing and pervasive.
  • The Rise of Agentic AI and Shadow AI: The emergence of agentic AI, where AI systems function as autonomous "employees," introduces novel security challenges. Organizations are already deploying AI employees, raising concerns about their security awareness training and access controls. Internally, shadow AI—employees using unsanctioned AI tools for efficiency—poses a substantial risk, exposing sensitive corporate data and bypassing established governance frameworks. McQuiggan notes that 83% of organizations plan to deploy agentic AI, but only a third feel confident in securing it.
  • Emerging Attack Vectors: Digital Arrests and Synthetic Identities: Beyond financial fraud, AI is enabling new forms of coercion. McQuiggan highlights digital arrest scams prevalent in India, where deepfake video calls simulating law enforcement, courtrooms, and judges are used to psychologically pressure victims into making urgent payments to avoid "arrest." Additionally, synthetic identities and face-swapping technology are being used by nation-state actors and malicious individuals to infiltrate companies, particularly in tech, during hiring processes.
  • Human Inability to Reliably Detect Deepfakes: A UC Berkeley study involving Sora 2 (OpenAI), VO31 (Google), and C Dance (ByteDance) revealed that even after education and repeated viewing, participants could only reliably detect deepfakes approximately 60% of the time with longer viewing periods. Sora 2 was found to be the least detectable. Humans primarily look for unnatural movement, anomalies in hands, and facial inconsistencies, but these artifacts are rapidly being refined by advanced AI models. This underscores the futility of relying solely on human detection.
  • The Inadequacy of Current Deepfake Detection Tools: While numerous enterprise-grade deepfake detection tools exist (e.g., Get Real, Valyia, Sensity, Hypervisor, Deepware), they often produce a high rate of false positives, incorrectly identifying real videos as fake. McQuiggan's own deepfake detector tool, for example, classified his opening deepfake as "real," indicating that current technological solutions struggle to keep pace with the rapid advancements in deepfake generation.

Technical Deep Dive

▶ Watch: Emerging digital arrest scams via deepfake video (25:30)

The technical core of AI deepfake generation, as demonstrated by James McQuiggan, involves a synergistic combination of readily available tools, APIs, and a streamlined workflow. The process, which McQuiggan has optimized to under four minutes with his custom Fakemaker application (spelled "Phakemaker" in his slides, but pronounced "Fakemaker"), leverages several key components:

  1. Source Material Acquisition: The initial step involves obtaining a video of the target individual. This is typically done by downloading existing content from platforms like YouTube, where public figures, including CEOs, often have extensive digital footprints from interviews, presentations, or social media posts. For his opening deepfake of Michelle, McQuiggan used one of the keynote speaker's YouTube videos.
  2. Audio Extraction and Synthesis:
  • Once the video is acquired, the original audio is separated.
  • A critical component is ElevenLabs, an AI-powered voice synthesis platform. This service allows users to create a voice model of the target individual by providing a short audio sample (as little as 10 seconds).
  • After the model is built, the desired script (the new message the deepfake will convey) is fed into ElevenLabs, which then generates a realistic audio file in the target's voice. This audio file is the foundation for the new deepfake dialogue.
  1. Lip-Syncing and Video Generation:
  • The generated audio file is then combined with the original video using specialized lip-syncing software. McQuiggan specifically highlights SyncSo (sync.so) as his preferred tool, though he also mentions Lemon Slice and Hedra.
  • Unlike older deepfake methods that might generate an entirely new video from a single image, SyncSo is designed to take an existing video and precisely adjust the mouth area to synchronize with the new audio track. This approach maintains the original video's quality, background, and body language, making the deepfake more seamless and harder to detect than a fully generated video. The process of generating the final video typically takes the longest, around five to six minutes, even with optimized tools.
  • McQuiggan's Fakemaker application (not publicly available, but a personal tool) integrates these steps, automating the download, audio processing via ElevenLabs, and lip-syncing with SyncSo, significantly reducing manual effort and processing time.

Beyond simple lip-syncing, the talk touches upon more advanced deepfake capabilities:

  • Face Swapping: Demonstrated using examples with Ira Winkler and Robert Downey Jr., this technique involves overlaying one person's face onto another's body in a video. While often detectable by "seams" around the face, especially when the head turns 90 degrees, continuous improvements make these harder to spot.
  • Full-Body Avatars: McQuiggan showcases a live demonstration using Daycart.ai, a commercial service that leverages API capabilities to create real-time, full-body avatars. This technology captures the speaker's movements and applies them to a chosen avatar, enabling highly convincing virtual presence in video calls. This moves beyond just facial manipulation to encompass entire body language, adding another layer of realism and threat. These services typically charge based on video duration (e.g., $5 for a couple of minutes of video).
  • Hardware Requirements: The computational power needed for creating these deepfakes is surprisingly accessible. McQuiggan notes that a $5,000 laptop equipped with a high-end video graphics card, such as an Nvidia RTX 5090 with 24GB of NV RAM, is more than sufficient to execute these sophisticated attacks. This low barrier to entry for hardware further democratizes deepfake creation for malicious actors.
  • AI Models and Attack Amplification: The talk references various AI models, including ChatGPT, Claude, Gemini, Grok, and specialized "dark web" LLMs like Worm GPT and XXX GPT (which lack ethical guardrails). These LLMs are used not only for crafting highly persuasive phishing emails (making them 14x more likely to succeed) but also for generating malicious code, bypassing security filters, and executing prompt injection attacks to manipulate AI systems for nefarious purposes. The mention of LiteLLM in a supply chain attack highlights the versatility of these tools in different cyberattack methodologies.
  • Agentic AI: This refers to AI systems that can autonomously perform tasks, make decisions, and interact with other systems, effectively acting as "employees." The discussion around AI Bill of Materials (AI BOMs) for agentic AI, similar to Software Bill of Materials (SBOMs), underscores the need for new security frameworks to manage and audit these advanced AI entities.

The technical landscape of deepfakes and AI-powered social engineering is characterized by rapid innovation, increasing accessibility, and a constant cat-and-mouse game between creators and detectors, demanding a proactive and multi-faceted defensive approach.

Demo / Proof of Concept

▶ Watch: Shift from spotting deepfakes to verification (46:50)

James McQuiggan effectively grounds his theoretical discussion in practical demonstrations, showcasing the alarming ease and sophistication of deepfake creation.

The talk immediately begins with a compelling deepfake of Michelle, the CypherCon keynote speaker. This pre-recorded video features a digitally manipulated Michelle delivering the opening remarks for McQuiggan's own presentation, complete with perfect lip-syncing and natural vocal delivery. The critical reveal is that this deepfake was created by McQuiggan in under four minutes using his custom tool, Fakemaker. This instant, real-world example serves as a powerful testament to the accessibility and efficiency of modern deepfake technology, setting the stage for the entire discussion.

McQuiggan then walks the audience through the general process of deepfake creation, highlighting how he uses ElevenLabs for voice generation and SyncSo for lip-syncing. He also shares examples of other deepfakes he has created:

  • A video of Elise at Cactus Con, which was generated using Lemon Slice from a single image.
  • A segment featuring Ira Winkler, a well-known Chief Information Security Officer and founder of Cruise Con. McQuiggan showed a side-by-side comparison of Ira Winkler's real video and a deepfake he created using lip-syncing software.
  • A demonstration of face-swapping software where he swapped his face with Ira Winkler's and then with Robert Downey Jr.'s. During this demonstration, he performed common deepfake detection tests, such as turning his head 90 degrees (to expose potential "seams") and placing his hand over his mouth. While acknowledging that these manual detection methods are becoming less effective as the technology advances, the demonstration illustrates the visual nature of these manipulations.

A significant part of the live demo involved McQuiggan attempting to create a new deepfake on stage using his Fakemaker application. He aimed to deepfake a video of "Monster" from Secret Con, inputting a YouTube link and a custom script: "Hello everyone. Thanks for coming into Secret Con. Want you all to know that James McQuiggan is the person to get when you want Cyber Dad jokes." This live attempt, while ambitious, encountered a technical hiccup—the application choked on the video download, likely due to a slow hotspot connection. Despite the technical difficulty, the attempt itself underscored the real-time nature of the tools and the straightforward input process.

Furthermore, McQuiggan showcased the capability for full-body avatars using Daycart.ai. He went live with his camera, stepping back to demonstrate how the service could track his full body movements and apply them to an avatar in real-time. This highlights the evolution beyond just facial deepfakes to full physical impersonation, making virtual interactions potentially indistinguishable from reality.

Finally, a pre-recorded video generated with Sora (OpenAI's text-to-video model) was shown, featuring a deepfake of McQuiggan telling dad jokes. This example further illustrates the capabilities of cutting-edge generative AI in creating entirely synthetic video content.

Collectively, these demonstrations serve as a powerful proof of concept, illustrating not only the current state of deepfake technology but also its rapid evolution and the urgent need for robust defensive strategies.

Defensive Implications

▶ Watch: Key actions: Audit AI, executive digital footprint, tabletop exercises (54:50)

The escalating threat of AI-powered deepfakes and social engineering demands a fundamental shift in defensive strategies, moving beyond traditional security paradigms. James McQuiggan outlines a comprehensive approach centered on human resilience, organizational readiness, and the strategic application of AI in defense.

Shifting from Detection to Verification

McQuiggan's most critical defensive implication is to "stop training folks to spot deepfakes" and instead focus on "educating them to verify." Given the increasing sophistication of deepfake technology, relying on human eyes to detect subtle anomalies is becoming futile, as evidenced by studies showing low detection rates. The emphasis must shift to a "trust and verify" mindset, encouraging polite paranoia and skepticism.

Practical verification techniques include:

  • Asking specific, personal questions: As seen in the Ferrari incident, asking questions only the legitimate individual would know (e.g., "What was the title of that book you lent me?") can quickly expose an imposter.
  • Out-of-band verification: If a request comes via an unusual channel (e.g., a video call from the CEO's new number), verify it through a known, trusted channel (e.g., a text message to their known number, or a call to their assistant).
  • Observing the context: While harder to detect, looking for unnatural eye movement, lighting inconsistencies, or jittery movements can still be clues, but should not be the primary defense.
  • Questioning the urgency and secrecy: Scammers often demand immediate action and secrecy ("Don't tell anyone"), which are red flags that should prompt verification.

Organizational Readiness and AI Governance

Organizations must adapt their internal processes and policies to mitigate AI-driven risks:

  • AI Governance Frameworks: Implement robust frameworks to manage AI risks. McQuiggan highlights several existing resources:
  • NIST AI Risk Management Framework (AI RMF)
  • ISO 42001 (AI Management System standard)
  • OWASP Large Language Model Top 10 and the emerging Agentic AI Top 10
  • MITRE Atlas for mapping AI-related attacks
  • Cloud Security Alliance (CSA) Framework for cloud-based AI deployments
  • Accredited A1 AI Audit and Certification
  • Visibility and Inventory: Organizations cannot protect what they don't know. It's crucial to gain full visibility into all AI tools and agents operating within the network, creating an AI inventory or AI Bill of Materials (AI BOMs). This is particularly vital for combating shadow AI, where employees use unsanctioned AI tools that can expose sensitive data.
  • Zero Trust for AI Agents: Apply Zero Trust principles to AI agents, enforcing least user privilege. AI agents should only have access to the data and systems absolutely necessary for their function, preventing them from inadvertently (or maliciously) exposing information or performing unauthorized actions.
  • Human-in-the-Loop: For critical decisions, maintain a human in the loop to prevent AI systems from making catastrophic errors (e.g., an AI optimizing a database by deleting it).
  • Incident Response Playbooks: Develop and update incident response playbooks specifically for AI-related incidents, including deepfake attacks, agentic AI failures, and synthetic media threats.
  • Cross-Functional Literacy: Security professionals and leaders must develop the ability to explain complex AI risks to both technical teams and executive leadership in understandable terms, articulating the business impact and ROI of security measures.
  • Upskilling Strategies: Continuously upskill security teams to keep pace with the rapidly evolving AI threat landscape.

Executive Awareness and Tabletop Exercises

  • Executive Digital Footprint Awareness: Educate executives about their public digital footprint (podcasts, videos, interviews) and how easily their voice and likeness can be cloned (as little as 10 seconds of audio). This awareness can help them understand the personal risk of impersonation.
  • Tabletop Exercises: Incorporate AI deepfake and social engineering scenarios into regular tabletop exercises. Crucially, involve non-IT departments like HR and the help desk, as they are often initial points of contact for such scams.

The Role of Technology in Defense

While human detection is flawed, technology still plays a role:

  • Deepfake Detection Tools: Acknowledge the existence of deepfake detection tools (e.g., Get Real, Valyia, Sensity, Hypervisor, Deepware). However, McQuiggan cautions that many are enterprise-focused, not widely available for social media, and suffer from high false positive rates, making them unreliable as a primary defense.
  • AI-Powered Defense: Leverage AI to defend against AI. This includes AI-driven anomaly detection, threat intelligence, and security automation, although specific tools were not detailed beyond Cisco's offerings.

Ultimately, defending against AI-powered deepfakes is not about finding a magic bullet but about integrating technological safeguards with a skeptical, verification-focused human element, reinforced by robust governance and continuous education.

Key Takeaways

  • Deepfakes are now fast, cheap, and highly effective: Modern AI tools enable the creation of convincing deepfake videos in under four minutes with minimal hardware investment (e.g., a $5,000 GPU). This low barrier to entry significantly lowers the cost of sophisticated attacks for cybercriminals.
  • AI amplifies existing social engineering threats: Large Language Models (LLMs) make phishing emails 14 times more likely to succeed, and deepfake audio/video elevates CEO impersonation scams to an unprecedented level of realism, leading to multi-million dollar losses.
  • Shift from "spotting" to "verifying": Given the increasing sophistication of deepfakes, human detection is unreliable. Organizations must train employees to verify suspicious requests, especially from authority figures, using out-of-band communication and asking specific, personal questions.
  • Agentic AI and Shadow AI introduce new internal risks: AI systems acting as autonomous "employees" (agentic AI) require new security models and governance frameworks. Unsanctioned use of AI tools by employees (shadow AI) poses significant data exposure and compliance risks.
  • Comprehensive AI governance and incident response are critical: Implementing AI governance frameworks (NIST, ISO, OWASP, MITRE Atlas), establishing AI inventories, applying Zero Trust principles to AI agents, and integrating AI-specific scenarios into incident response playbooks are essential for organizational resilience.
  • Educate executives and involve HR/Help Desk in defense: Executives are prime targets due to their public digital footprint. They need awareness of deepfake risks, and HR and help desk teams must be included in tabletop exercises to prepare for deepfake-driven social engineering attempts.

About the Speaker(s)

James McQuiggan is a seasoned cybersecurity expert and the founder of Apparent Security LLC, a venture dedicated to making cyber threats more transparent and understandable for organizations. With an extensive career spanning over two decades, James brings a wealth of experience across various facets of cybersecurity.

He spent 18 years at Siemens, a global technology powerhouse, where he held diverse roles encompassing networking, security awareness, phishing assessments, incident response, and product security. This broad foundation provided him with a holistic understanding of enterprise security challenges.

For the past six years, James has been a prominent voice in thought leadership at KnowBe4, a leading security awareness training and simulated phishing platform. During this time, he has delivered numerous presentations globally, with a particular focus on the evolving threats posed by AI and deepfakes. His passion for these topics and his ability to demystify complex security concepts make him a highly engaging and informative speaker.

Reviews

Dr. Zero (Offensive Security Researcher) — WEAK

This is a corporate awareness training module disguised as a conference talk. Gluing ElevenLabs and SyncSo APIs together in a script doesn't make you a researcher, and telling people to 'verify out of band' is advice we've been giving since the Nigerian Prince days.

Heather Calloway (CISO) — STRONG ACCEPT

A pragmatic, reality-based look at AI-enabled executive impersonation that correctly identifies the solution as process, not technology. The speaker argues effectively that training employees to visually spot deepfakes is a losing battle, making this highly relevant for your risk committee and finance leadership.

→ Top-rated talks at CypherCon 9.0 (2026)

All talks from CypherCon 9.0 (2026)