Cheap Tricks, High Impact: How AI is Powering Sloppy Influence Campaigns

Cheap Tricks, High Impact: How AI is Powering Sloppy Influence Campaigns

Speakers: Dina Sadek, Senior Analyst, Graphika; Margot Fulde-Hardy, Analyst, Graphika

Conference: CYBERWARCON

YouTube: https://www.youtube.com/watch?v=i_POSv8ud9U

Overview

In an era saturated with predictions about the transformative power of artificial intelligence, Dina Sadek and Margot Fulde-Hardy from Graphika presented a compelling counter-narrative at CYBERWARCON. Their talk, "Cheap Tricks, High Impact: How AI is Powering Sloppy Influence Campaigns," dissected the surprising reality of AI's current role in influence operations (IOs). Contrary to widespread fears of hyper-sophisticated, undetectable AI-driven propaganda, Sadek and Fulde-Hardy revealed that the dominant trend is the proliferation of "AI slobs" – low-quality, mass-produced, and often easily detectable AI-generated content.

This phenomenon, while seemingly benign or even laughable at first glance with examples like "shrimp Jesus" or "Buff Jim Cats," is being weaponized by state and non-state actors alike to scale influence campaigns with unprecedented ease and minimal cost. The talk highlighted how AI is primarily leveraged for impersonation, massive content generation, and the dissemination of polarizing, sensationalist material. Despite the inherent sloppiness of these campaigns, they pose a significant challenge to information integrity, not least because the very act of researching and reporting on them can inadvertently amplify their reach by feeding social media algorithms. The speakers urged the cybersecurity community to look beyond the hype and focus on the practical implications of these "cheap tricks," recognizing them as a learning ground for adversaries who are constantly adapting their tactics.

Background

▶ Watch: Introduction and the concept of 'AI slob' (0:00)

The advent of generative AI technologies in recent years sparked widespread speculation about their potential to revolutionize influence operations. In early 2023, prominent institutions like Georgetown University, OpenAI, and the Stanford Internet Observatory jointly predicted that AI would profoundly impact the actor-behavior-content (ABC) framework of IOs. They anticipated an increase in the number of actors and outsourcing, a dramatic scaling of campaigns, the emergence of novel and efficient tactics, and a significant boost in the credibility and persuasiveness of messages, coupled with a decrease in the discoverability of state-sponsored messaging. These predictions fueled concerns that AI would usher in an era of undetectable, highly sophisticated disinformation.

As 2024, a year marked by an unprecedented number of global elections where 60% of the world's population cast their votes, unfolded, industry players echoed these warnings. Anthropic, for instance, explicitly stated its expectation of "unanticipated uses of AI." However, by the end of 2023, a consensus began to emerge among many industry observers: the immediate threat posed by generative AI in influence operations might have been "overhyped." Initial assessments suggested that AI-generated activity had not been significant, nor had it demonstrably impacted elections.

Despite this recognition, fears surrounding AI's potential in 2025 and beyond have not dissipated. Leaks, such as those concerning Galaxi – a Chinese company with alleged ties to the Chinese military – claiming to integrate AI for microtargeting campaigns, underscore persistent anxieties about future capabilities, even if concrete evidence remains elusive. Graphika's research, however, painted a different picture of the current landscape. Instead of sophisticated, covert AI operations, what has predominantly materialized are "sloppy, mass-produced AI-generated content and infrastructure." This "AI slob" content, defined as low-quality AI-generated material designed for virality, ranges from humorous oddities like "shrimp Jesus" to more insidious, politically charged fabrications. The problem exists because generative AI tools have lowered the barrier to entry for creating vast amounts of content, making it cheap, fast, and scalable for actors with malicious intent, even if the quality is poor. This shift forces a re-evaluation of how we perceive and combat AI-powered influence.

Key Findings

▶ Watch: How AI is actually used in sloppy influence campaigns (4:40)

The Graphika presentation meticulously detailed the primary ways AI is currently being leveraged in influence operations, challenging the prevailing narrative of AI leading to insurmountable sophistication. Their key findings indicate that AI's utility in IOs is less about crafting flawless, deepfake-level deception and more about brute-force scalability and low-cost content generation.

Firstly, AI is predominantly used to impersonate trusted media sources and politicians. This involves creating fake profiles, generating artificial anchors for videos, or crafting voiceovers that mimic legitimate public figures. The goal is to lend an air of false authority or familiarity to the disseminated narratives, even if the execution is visibly flawed.

Secondly, and critically, AI serves to massively increase the scale of these campaigns. With generative tools, operators can produce thousands of posts, comments, videos, and images in a fraction of the time and cost it would take human operators. This enables adversaries to flood social media platforms with their messaging, attempting to overwhelm legitimate discourse through sheer volume. The speakers noted that campaigns like "Sneer Review" demonstrate this by conducting multiple, often contradictory, campaigns simultaneously, repeating content weekly, daily, and even minute by minute with consistent hashtags. This suggests a commercialized approach where generative AI is used to service multiple clients or pursue diverse objectives efficiently.

Thirdly, the content produced is overwhelmingly highly polarizing, spammy, and sensationalist. The strategy appears to be driven by the notion that "all publicity is good publicity." By generating outrageous or emotionally charged content, even if overtly fake, operators aim to capture attention, provoke engagement, and push divisive narratives. The examples from Operation Overload, which promoted narratives of Moldova becoming a "sex tourism center for pedophiles" or its president having "psychological issues due to hormonal imbalance," illustrate this pursuit of sensationalism.

A crucial, and somewhat ironic, finding was the unintended role of researchers and the broader cybersecurity community in amplifying these campaigns. While the AI-generated content often receives minimal initial engagement or views on social media platforms, its discovery and subsequent reporting by researchers inadvertently feed into platform recommendation algorithms. This analytical attention provides the campaigns with visibility and exposure they might not otherwise achieve, effectively helping the influence operations achieve their objectives.

Finally, the speakers emphasized that despite the current "sloppy" nature of AI-generated content, operators are actively learning and adapting. They are observing how their techniques are detected and reported, and are likely refining their methods. This implies that the current state of "sloppy AI" is temporary; it represents a testing ground for adversaries who will undoubtedly leverage these lessons to develop more sophisticated and effective tactics in the future. The core message is that the ease and low cost of AI-powered content generation make these operations "quick and easy," "scalable," and achievable with "minimal effort," thereby ensuring their continued proliferation and evolution.

Technical Deep Dive

▶ Watch: Case study: Spamouflage's AI-generated personas (5:00)

The technical deep dive into AI-powered influence operations reveals a landscape where adversaries prioritize quantity and low cost over pristine quality, yet their methods demonstrate a clear progression in leveraging available AI tools. Graphika's analysis of publicly documented IOs, attributed to both state and non-state actors, highlights several key Tactics, Techniques, and Procedures (TTPs).

Early applications, particularly observed in Spamouflage, a large pro-China influence operation, involved the use of Generative Adversarial Networks (GANs) to create synthetic profile pictures. These AI-generated faces, often uncanny but generally believable at first glance, provided a cheap and scalable way to populate fake social media accounts with seemingly authentic personas, thereby enhancing the credibility of the narratives being pushed. Spamouflage's evolution continued into 2023 with the use of AI video creation platforms, notably from a UK-based company, to generate videos featuring AI-generated anchors. These synthetic presenters delivered pro-China narratives, attempting to mimic legitimate news broadcasts. By 2024, ahead of the US elections, Spamouflage further adapted by leveraging AI-generated avatars as narrators in videos discussing US politics, sometimes even creating linked X (formerly Twitter) accounts for these avatars to bolster their perceived legitimacy. However, these efforts were often easily detectable due to clear flaws, such as mismatched body and lip movements, and numerous grammatical errors in the scripts, revealing the limitations of the technology or the operators' haste.

Another significant TTP is the use of large language models like ChatGPT for generating social media comments. The "Sneer Review" operation, likely originating in China and first exposed by OpenAI in June 2025 (a potential typo in transcript, likely 2023 or 2024 given context), exemplifies this. Its primary TTP involves using ChatGPT to generate and post social media comments across various platforms and websites. Graphika's monitoring revealed that Sneer Review conducts multiple campaigns concurrently, often with contradictory stances, strongly suggesting a commercial actor utilizing generative AI to serve diverse clients. These campaigns are characterized by their repetitive nature, cycling weekly, day after day, and minute after minute, often employing the same hashtags to flood platforms. While pushed at scale, these campaigns typically remain within their own "echo chambers," only gaining wider traction when researchers report on them.

The pro-Russia influence campaigns, Operation Overload and Operation Undercut, demonstrate further applications of AI. Operation Overload focuses on AI-generated voiceovers to impersonate European authorities, fact-checkers, journalists, and politicians. These voiceovers, while improving in quality, are used to disseminate pro-Russian narratives, often targeting European countries during elections, as seen in Moldova with sensationalist claims. Operation Undercut complements this by employing AI-enhanced videos and images to spread similar pro-Russian narratives. Both operations share a common trait: despite the use of AI, their content rarely breaks out of their self-contained echo chambers. A notable exception was one video targeting the US that achieved significant amplification on mainstream social media platforms, indicating that while overall reach may be limited, specific pieces of content can occasionally resonate more broadly.

The overarching technical theme is the scalability and efficiency that AI provides. Adversaries can generate vast amounts of content "by the press of a button," making it "easily scalable and distributable" with "very minimal effort." This ease of production allows for a broad deployment of "highly polarizing, spammy, sensationalist content," operating under the principle that "all publicity is good publicity." While the technical quality of the AI output often remains "sloppy," the sheer volume and low cost of generation make these campaigns a persistent and evolving threat. The inherent flaws, such as grammatical errors or unnatural video movements, serve as crucial "AI fingerprints" that can aid in their detection and attribution, a point emphasized by the speakers as a critical avenue for future investigations.

Demo / Proof of Concept

▶ Watch: Unintended impacts of researchers on IO amplification (9:00)

The speakers did not present a live demonstration or proof of concept during their talk. Instead, they focused on presenting their research findings through a series of case studies and observations of existing influence operations that are actively leveraging AI technologies in the wild.

Defensive Implications

▶ Watch: Why investigating sloppy AI influence campaigns is still crucial (10:20)

The insights presented by Dina Sadek and Margot Fulde-Hardy carry significant defensive implications for cybersecurity professionals, social media platforms, and the research community. The current landscape of "sloppy AI" influence operations, while not as sophisticated as once feared, necessitates a recalibration of defensive strategies.

Firstly, for social media platforms and content moderators, the focus should expand beyond just detecting individual pieces of AI-generated content. The speakers stressed the importance of investigating the proxies and distribution networks that these influence operations utilize. Identifying the infrastructure, accounts, and propagation methods behind the low-quality content can be more effective than a never-ending battle against individual AI artifacts. Platforms should enhance their capabilities to detect patterns of mass content generation, repetitive posting behaviors, and the coordinated use of AI-generated personas, even if the content itself is flawed. This includes developing robust detection mechanisms for common "AI fingerprints" like grammatical errors, unnatural voice inflections, or synchronized, low-engagement posting.

Secondly, for researchers and analysts, a critical defensive implication is to be acutely aware of the "researcher amplification" effect. While documenting and exposing influence operations is vital, the act of reporting on these campaigns, particularly those with low initial engagement, can inadvertently feed social media algorithms, granting the content increased visibility and validating the adversaries' efforts. Researchers should consider strategies for responsible disclosure that minimize this unintended amplification, perhaps by focusing on the underlying infrastructure and TTPs rather than solely highlighting the most egregious examples of AI-generated content. Shifting the investigative paradigm from content analysis to network and operational analysis can help mitigate this unintended consequence.

Thirdly, the talk underscores the need for proactive defense against evolving threats. The current "sloppiness" of AI-generated content is a temporary state. Adversaries are learning and adapting their techniques based on detection and reporting. Defenders must anticipate this evolution, investing in research and development for detecting more sophisticated AI-generated content, including improved deepfakes, more natural language generation, and advanced persona management. This includes understanding how operators are experimenting with different AI models, prompt engineering, and content blending to bypass current detection methods.

Finally, the very nature of "cheap tricks, high impact" suggests that defenders should not dismiss low-quality AI content as inconsequential. Its ease of production, scalability, and ability to test narratives make it a valuable tool for adversaries. Defenders must recognize that the strategic value for operators lies in the low cost and high volume, enabling them to sustain campaigns over long periods and across numerous platforms. Therefore, defensive strategies must prioritize disrupting the economic and operational viability of these campaigns, making it more expensive and difficult for actors to leverage AI for influence, regardless of the output quality. This could involve targeting the AI service providers, identifying and blocking large clusters of coordinated inauthentic behavior, and educating the public on the characteristics of AI-generated disinformation.

Key Takeaways

• AI's current role in influence operations is largely "sloppy" and focused on mass production, not sophisticated deception. Initial predictions of hyper-realistic, undetectable AI-driven propaganda have not yet materialized; instead, adversaries prioritize quantity and low cost.
• AI is primarily used for impersonation, massive campaign scaling, and generating polarizing, sensationalist content. This allows actors to flood platforms with narratives using minimal effort and expense, operating on the principle that "all publicity is good publicity."
• "AI fingerprints" like grammatical errors, mismatched lip movements, and repetitive content patterns are common and aid in detection. While AI quality is improving, current methods often leave observable flaws that defenders can leverage for identification.
• Researchers and analysts inadvertently amplify low-engagement campaigns. By reporting on AI-generated influence operations, the research community can feed algorithms and provide visibility, thereby helping adversaries achieve their objectives.
• Adversaries are actively learning and adapting their AI-powered tactics. The current "sloppiness" is a temporary phase; operators are refining their techniques, necessitating a proactive and evolving defensive posture.
• Defensive efforts should focus on distribution networks and underlying infrastructure, not just individual pieces of content. Investigating the proxies and operational networks behind these campaigns, alongside identifying AI fingerprints, offers a more sustainable defense strategy.

About the Speaker(s)

Dina Sadek is a Senior Analyst on the Graphika intelligence team. Her work focuses extensively on influence operations, examining their impact and methodologies within both electoral and non-electoral contexts.

Margot Fulde-Hardy is an Analyst at Graphika. Her research primarily delves into the activities of Chinese actors in the information space and issues related to electoral integrity.