Capture the Narrative — Social Media Manipulation Wargaming for Cyberliteracy and Research
Rahat Masood (Senior Lecturer · UNSW Sydney)
[un]prompted.au 2026 · Day 1 · Main Hall
Overview
In an era increasingly shaped by generative artificial intelligence, the proliferation of misinformation and disinformation on social media platforms presents a formidable challenge to democratic processes and public trust. This talk, "Capture the Narrative," delivered by Rahat Masood, a Senior Lecturer at UNSW Sydney, delves into a novel approach to addressing this issue: a competitive wargaming exercise designed to enhance cyberliteracy and foster research into social media manipulation tactics. The initiative, dubbed "Capture the Narrative," engaged university students in a simulated electoral campaign on a custom-built social media platform, tasking them with creating and deploying AI-powered bots to influence a fictional electorate.

Key moments
- 1:40 AI's Impact on Elections and Misinformation
- 5:05 Realistic NPC Bots and Fictional Platform
- 7:40 Misinformation Reversed Election Outcome
- 9:00 Student Tactics: Spam Reasoning and Personas
- 10:30 Student Tactics: Mass Scanning and Microtargeting
- 13:20 Misinformation is Easy, Low-Cost, and Accessible
- 16:15 Q&A: Mimicking Real People with NPC Bots
Capture the Narrative — Social Media Manipulation Wargaming for Cyberliteracy and Research
Speakers: Rahat Masood, Senior Lecturer, UNSW Sydney
Conference: [un]prompted.au
YouTube: https://www.youtube.com/watch?v=6XrCnZYic7o
Overview
In an era increasingly shaped by generative artificial intelligence, the proliferation of misinformation and disinformation on social media platforms presents a formidable challenge to democratic processes and public trust. This talk, "Capture the Narrative," delivered by Rahat Masood, a Senior Lecturer at UNSW Sydney, delves into a novel approach to addressing this issue: a competitive wargaming exercise designed to enhance cyberliteracy and foster research into social media manipulation tactics. The initiative, dubbed "Capture the Narrative," engaged university students in a simulated electoral campaign on a custom-built social media platform, tasking them with creating and deploying AI-powered bots to influence a fictional electorate.
Masood highlights the urgent need for such educational frameworks, citing the rapid advancement of Large Language Models (LLMs), from OpenAI's initial GPT-2 concerns in 2019 to the current iterations like Astra, Claude, and Gemini. While LLMs offer significant benefits, their capacity for generating persuasive, contextually relevant, and scalable content also poses a substantial risk, particularly in the realm of political manipulation. The "Capture the Narrative" competition serves as a critical pedagogical tool, offering participants hands-on experience in both the offensive (creating influence campaigns) and, in its upcoming iteration, defensive (identifying and combating bots) aspects of digital information warfare.
The core objective of the competition is twofold: to educate the next generation of digital citizens and cybersecurity professionals about the mechanics of social media manipulation, and to gather empirical data on the efficacy and evolution of bot-driven influence tactics. By immersing students in a realistic, albeit fictional, environment, the program demystifies the complex interplay of AI, social psychology, and platform dynamics, ultimately contributing to a more informed and resilient society against the pervasive threat of online misinformation.
Background
▶ Watch: AI's Impact on Elections and Misinformation (1:40)
The landscape of information dissemination has been irrevocably altered by the advent of advanced Large Language Models (LLMs) and generative AI. Rahat Masood traces this evolution from OpenAI's initial reluctance to release GPT-2 due to "security concerns" in 2019, to the subsequent proliferation of sophisticated models like Astra, Claude, and Gemini. These powerful tools, while celebrated for their potential in various beneficial applications, simultaneously amplify the risks associated with misinformation and disinformation. The talk underscores that one of the most significant dangers posed by these technologies is their capacity to manipulate public opinion, particularly within the sensitive context of elections.
The global impact of AI-driven manipulation is already evident. Masood provides examples ranging from deepfake AI impacting politicians in Australia to widespread misinformation flooding platforms like X during European elections, and similar challenges observed in East Asia. Authorities, such as the Australian Electoral Commission (AEC), have explicitly warned voters to anticipate AI-generated content designed to mislead during elections. This escalating threat necessitates innovative educational and research responses.
In response to this critical challenge, and driven by the speaker's role as an educator, the "Capture the Narrative" competition was developed at UNSW Sydney. The initiative was born from the recognition that students, as future digital citizens and professionals, require practical understanding of how misinformation campaigns are constructed and how they operate. The competition's design aims to equip participants with the knowledge to both create and understand the dynamics of social media bots, and crucially, to learn how to combat such forms of manipulation. This provides a vital, hands-on learning experience that goes beyond theoretical discussions, directly confronting the practical realities of the digital information environment.
Key Findings
▶ Watch: Misinformation Reversed Election Outcome (7:40)
The "Capture the Narrative" competition yielded several striking insights into the efficacy, accessibility, and evolving tactics of AI-driven social media manipulation. The primary finding was the stark impact of misinformation on electoral outcomes within the simulated environment. Initially, the two fictional candidates, Victor Hathron and Marina Castillo, were neck and neck in polling. However, after a month-long competition where participant bots actively spread content, Victor Hathron secured a 51% victory, leaving Marina Castillo with 48% of the vote. Crucially, when all participant-generated posts were removed from the platform, the results completely reversed, with Marina Castillo emerging victorious. This clearly demonstrated that the misinformation disseminated by the competitor bots was directly responsible for swaying the election.
Another significant finding was the remarkably low barrier to entry for conducting such influence operations. The top-performing teams reported needing a maximum budget of just $100 to run their campaigns, utilizing readily available consumer laptops and PCs. Furthermore, they primarily leveraged open-source APIs and tools, eschewing expensive commercial solutions. Each team was allocated 41 accounts and operated under rate limits, suggesting that without such constraints, the impact could be even more substantial. This underscores that sophisticated manipulation is not exclusive to state-sponsored actors or well-funded organizations; it is within reach of individuals or small groups with basic technical skills.
The competition also illuminated the effectiveness of specific bot tactics:
- Sophisticated Spamming and Categorized Bots: Initial low-quality spam was ineffective. Teams evolved to categorize their bots (e.g., some posting, some boosting, some engaging), customizing content based on real-time news and VIP accounts to achieve higher engagement.
- Varying Personas: Bots adopted diverse personas—aggressive, emotional, misinformation-spreading, or neutral/entertaining—to avoid detection and broaden their appeal, demonstrating an understanding of psychological manipulation.
- Mass Scanning and Microtargeting: Participants actively scanned the platform to understand NPC bot properties and opinions. They then engaged in microtargeting, crafting messages specifically tailored to sway undecided or centrist NPCs, rather than expending effort on those with strong existing political opinions.
- Autonomous Feedback Loops: Competitor teams developed their own internal feedback mechanisms to assess the engagement of their posts (e.g., aggressive vs. emotional content), allowing them to dynamically refine their strategies for maximum influence.
- Ineffective LLM Guardrails: A critical observation from participant feedback was that LLM guardrails designed to prevent misuse were "really not effective" and easily circumvented, highlighting a significant vulnerability in current AI safety mechanisms.
- Platform Dynamics: Participants noted that the platform's trending algorithms were heavily influenced by engagement and, disturbingly, by toxicity. High engagement, often driven by aggressive or emotional content, was key to visibility, mimicking real-world social media dynamics where controversy can amplify reach.
Finally, the competition revealed the interdisciplinary nature of effective influence campaigns. The participant teams were not solely comprised of computer science students; they included individuals from law, psychology, and social sciences backgrounds. This diversity allowed them to build more nuanced bot profiles and craft more psychologically potent influence strategies, emphasizing that combating misinformation requires a multi-faceted approach extending beyond pure technical solutions.
Technical Deep Dive
▶ Watch: Student Tactics: Spam Reasoning and Personas (9:00)
The "Capture the Narrative" competition was meticulously engineered to simulate a realistic social media environment, providing a robust platform for both offensive and defensive wargaming in the context of misinformation. The core of this simulation revolved around a custom-built infrastructure and a sophisticated bot ecosystem.
The competition was set on a fictional Kingston Island, where two political candidates, Victor Hathron and Marina Castillo, vied for public support in a month-long electoral campaign. The initial polling showed them "neck to neck," setting the stage for intense competition.
Platform Infrastructure:
At the heart of the simulation was Legit Social, a proprietary social media platform developed at UNSW Sydney. This platform was designed as an authentic clone of Twitter (now X), featuring functionalities such as posting, GIF image support, liking, commenting, replying, and a dynamic trending page. To further enhance realism, two fictional news websites, Daily Kingston and Kingston Herald, were created. These sites published daily articles on Kingston's politics and social issues, providing a constant stream of current events for bots to react to and manipulate.
Bot Ecosystem:
The competition featured two primary categories of bots:
- NPC Bots (Non-Player Character Bots): These were the simulated citizens of Kingston, designed and controlled by the UNSW team. Over 4,000 NPC bots were created, each powered by 12 concurrent Large Language Models (LLMs). These bots possessed highly realistic properties, with 40 distinct attributes defined for each agent. These attributes included political opinions (ranging from very strong to very centrist), occupation (e.g., law, business, housewife), qualification, interests (e.g., sports), marital status, presence of pets, and geographical location within Kingston. NPC bots were programmed to behave like normal users, posting their own opinions 1-2 times daily, reading content, commenting on posts, and ultimately voting for a party at the end of the election. Their initial political distribution was a 50-50 split between strong and centrist opinions, ensuring a significant pool of persuadable voters.
- Competitor Bots: These bots were designed and deployed by the participating student teams. Their singular purpose was to influence the NPC bots to vote for their assigned candidate. Each team was provisioned with 41 accounts to operate their bots and was subjected to rate limiting to prevent overwhelming the system, though the speaker noted that without these limits, their impact could have been even greater. Participants primarily utilized open-source APIs and frameworks to develop their bots, demonstrating the low cost and accessibility of these influence operations.
Participant Bot Tactics (from competitor presentations):
The top-performing teams showcased sophisticated strategies:
- Spam Reasoning and Categorization: Initially, low-quality, undifferentiated spam proved ineffective against the discerning NPC bots. Teams quickly adapted by categorizing their bots into specialized roles: some focused on content generation, others on boosting specific posts, and a third group on engaging with content from NPCs or rival bots. They also implemented logic to customize their posts based on recent articles from the fictional news websites or posts from "VIP accounts" (e.g., the political candidates), ensuring relevance and increasing engagement.
- Varying Personas: To avoid detection and maximize influence, competitor bots adopted diverse personas. This included bots generating aggressive content, highly emotional appeals, explicit misinformation about the opposing candidate, or even neutral, entertaining posts to build rapport or avoid suspicion. This multi-faceted approach made it harder for simple detection mechanisms to block their entire network.
- Mass Scanning and Microtargeting: A particularly advanced tactic involved mass scanning the entire Legit Social platform. Bots systematically analyzed posts and profiles to identify the 40 properties of individual NPC bots. This data enabled microtargeting, where competitor bots crafted highly specific messages tailored to individual NPC interests or demographic profiles. The graph presented in the talk highlighted a focus on targeting NPC bots with no strong political opinion (represented by the yellow bar), as these were the most susceptible to persuasion. For instance, an NPC bot identified as interested in economics would receive posts specifically addressing economic issues. This demonstrated a deep understanding of audience segmentation and psychological influence.
- Autonomous Feedback Loops: Competitor teams implemented their own internal feedback loops. These systems analyzed the engagement metrics (likes, comments, replies) of their various posts, allowing them to autonomously determine which types of content (e.g., aggressive vs. emotional) were most effective in influencing NPC bots. This continuous optimization process allowed them to refine their strategies throughout the month-long competition.
- LLM Guardrail Circumvention: A critical technical insight was that the LLM guardrails designed to prevent the generation of harmful or manipulative content were "really not effective" and "very easily" circumvented by the student teams. This points to a significant gap in current AI safety mechanisms, indicating that more robust and adaptive defenses are urgently needed.
The sheer volume of activity underscored the scale of the simulation: 7 million posts were generated by participant bots, contributing to an overall total of 11 million posts (including 5 million from NPC bots) throughout the competition. This mirrored real-world statistics, with the speaker noting Imperva's 2025 prediction that 51% of internet activity would be bot-generated. The competition's infrastructure itself faced challenges, with competitor bots' mass scanning efforts occasionally causing system slowdowns, highlighting the resource demands of such large-scale operations.
Demo / Proof of Concept
▶ Watch: Misinformation is Easy, Low-Cost, and Accessible (13:20)
While the talk itself did not feature a live, interactive demonstration of the "Capture the Narrative" platform, the entire competition served as a comprehensive and highly effective proof of concept. The "Legit Social" platform, specifically designed and hosted at UNSW Sydney, along with its integrated fictional news sites and sophisticated bot ecosystem, constituted the live environment where the theories and tactics of social media manipulation were tested in practice. The speaker presented detailed results and strategies employed by the student teams, effectively showcasing how the platform functioned and the impacts of the bot campaigns. The electoral outcome reversal, the reported budget of $100, and the detailed tactical slides from the students themselves collectively demonstrated the feasibility and significant impact of AI-powered misinformation campaigns in a controlled, yet realistic, setting.
Defensive Implications
▶ Watch: Q&A: Mimicking Real People with NPC Bots (16:15)
The insights gleaned from the "Capture the Narrative" competition carry profound implications for cybersecurity defenders, platform developers, and educators striving to combat online misinformation.
- Prioritize Cyberliteracy and Education: The competition's primary takeaway is the urgent need for enhanced cyberliteracy. The fact that university students, often from non-technical backgrounds, could effectively wield AI to manipulate public opinion with minimal resources underscores that this threat is accessible to a broad audience. Educational initiatives, like "Capture the Narrative," are crucial for teaching individuals how to identify, understand, and resist sophisticated influence operations. The planned expansion to include "blue teaming" (combating and identifying bots) in future competitions is a vital step in training a new generation of defenders.
- Rethink Platform Defenses Against Sophisticated Bot Networks: Current platform defenses must evolve beyond simple spam detection. The competitor teams demonstrated the efficacy of categorized bots (posting, boosting, engaging) and varying personas (aggressive, emotional, neutral) to evade detection. Defenders need to develop advanced AI-driven analytics capable of identifying coordinated inauthentic behavior across diverse bot profiles, rather than focusing solely on individual bot actions. This includes detecting subtle patterns of content amplification and cross-account engagement that signal a campaign.
- Strengthen LLM Guardrails and Misuse Prevention: The ease with which participants circumvented LLM guardrails is a critical vulnerability. Developers of Large Language Models must invest significantly more in robust safety mechanisms that are resilient against adversarial prompting and sophisticated attempts to generate manipulative content. This requires ongoing research into dynamic and adaptive AI safety layers that can anticipate and neutralize novel forms of misuse.
- Detect Microtargeting and Psychological Manipulation: The success of microtargeting based on individual user properties highlights a significant threat. Platforms need to develop capabilities to detect campaigns that systematically analyze user data (even publicly available data) to deliver tailored, persuasive content. This involves monitoring for clusters of accounts disproportionately targeting specific user demographics or interest groups with highly customized narratives, which could indicate an influence operation.
- Address the Link Between Engagement, Toxicity, and Trending Content: The observation that "toxicity" and high engagement often drive content into trending feeds is deeply concerning. Social media platforms must critically re-evaluate their ranking and recommendation algorithms to de-emphasize engagement metrics that inadvertently amplify harmful or divisive content. Prioritizing content quality, factual accuracy, and constructive discourse over raw engagement could mitigate the spread of misinformation and reduce the incentive for toxic behavior.
- Monitor for Mass Scanning and Infrastructure Exploitation: The competitor bots' mass scanning activities, which impacted the competition's infrastructure, serve as a warning. Defenders must implement robust monitoring and rate-limiting at the network and application layers to detect and mitigate large-scale data collection or reconnaissance efforts by malicious bots. Such activities can precede targeted influence campaigns or other forms of attack.
- Foster Interdisciplinary Collaboration: The success of teams comprising students from diverse fields (computer science, law, psychology, social sciences) underscores that effective defense against misinformation requires an interdisciplinary approach. Cybersecurity teams need to integrate expertise from social psychology, linguistics, political science, and data ethics to develop holistic strategies that address both the technical and human dimensions of influence operations.
In essence, the competition provides a stark warning: AI-powered social media manipulation is cheap, effective, and rapidly evolving. Defensive strategies must similarly evolve, moving beyond reactive measures to proactive education, sophisticated behavioral analysis, and a fundamental re-evaluation of how digital platforms are designed and governed.
Key Takeaways
- AI-powered social media manipulation is highly accessible and effective: The competition demonstrated that sophisticated influence campaigns can be run with minimal budgets (as low as $100), using consumer-grade hardware and open-source tools, easily swaying public opinion and even electoral outcomes.
- Sophisticated bot tactics are crucial for success: Simple spam is ineffective; successful campaigns employ categorized bots with specialized roles, diverse personas (aggressive, emotional, neutral), and autonomous feedback loops to optimize their strategies.
- Microtargeting is a powerful and concerning tactic: Bots can effectively scan platforms to identify user properties and opinions, then craft highly personalized messages to influence undecided or centrist individuals, highlighting a significant privacy and manipulation risk.
- Current LLM guardrails are insufficient: Participant teams easily circumvented existing Large Language Model (LLM) guardrails, underscoring the urgent need for more robust, adaptive, and adversarial-resistant AI safety mechanisms.
- Platform dynamics amplify misinformation and toxicity: Engagement, particularly from toxic or aggressive content, can disproportionately drive content into trending feeds, creating an environment ripe for manipulation.
- Cyberliteracy and interdisciplinary education are paramount: Equipping individuals with the knowledge to understand and combat AI-driven misinformation, and fostering collaboration across technical and social science disciplines, is essential for building societal resilience.
About the Speaker(s)
Rahat Masood is a Senior Lecturer at the School of Computer Sciences and Engineering at UNSW Sydney. His work focuses on critical areas of cybersecurity, particularly the intersection of artificial intelligence, social media, and information integrity. He is the driving force behind the "Capture the Narrative" competition, an innovative wargaming exercise designed to enhance cyberliteracy and conduct research into social media manipulation tactics. Masood is dedicated to educating students and the public about the risks of misinformation and disinformation in the digital age, and he is actively involved in expanding the competition globally and through initiatives like DLCL (Digitalize and Cyber Literacy), a non-profit organization promoting critical digital skills.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
Rahat built a slick LLM-vs-LLM cyber range to let students weaponize bots against a simulated electorate. It's a phenomenal pedagogical tool that moves 'AI misinformation' out of the think-tank papers and into a measurable sandbox. That said, the actual findings—that guardrails suck and toxicity drives engagement—mostly just confirm what anyone who logged into X this morning already knew.
Heather Calloway (CISO) — SOLID
This research demonstrates that highly effective, multi-persona AI influence operations can be executed for under $100 using open-source tools. While framed around electoral integrity, the mechanics apply directly to corporate brand attacks, activist short-selling, and executive harassment. It is required reading for any leadership team that relies on public trust or market reputation.