Exploring Dataset Diversity for GenAI Image Inpainting Localisation in Digital Forensics

Exploring Dataset Diversity for GenAI Image Inpainting Localisation in Digital Forensics

Speakers: Matthew Thomson, Doctoral researcher, Edinburgh Napier University

Conference: DFDS

YouTube: https://www.youtube.com/watch?v=LftBATL_ojc

Overview

In an era increasingly shaped by synthetic media, the ability to accurately detect and localize manipulations made by Generative Artificial Intelligence (GenAI) has become paramount for digital forensics. Matthew Thomson, a doctoral researcher at Edinburgh Napier University, presented his critical research on this topic, focusing specifically on GenAI image inpainting localization. His talk delves into the challenges posed by the lack of diverse datasets for training detection models and the practical difficulties faced by forensic practitioners in identifying manipulated regions within an image. Thomson's work aims to bridge the gap between academic research and real-world forensic application, striving to develop robust and understandable methodologies that can withstand scrutiny in legal contexts.

The presentation highlights the dual nature of GenAI—its benign applications in entertainment and its perilous potential for malicious use. From fabricating political events and spreading propaganda to enabling harassment and sophisticated cybercrime, GenAI-generated imagery can have profound societal and individual impacts. Thomson's research is particularly pertinent as it tackles the subtle yet powerful technique of image inpainting, where specific objects within an image are seamlessly removed or replaced. By examining dataset diversity and the performance of current detection algorithms, his work seeks to lay the groundwork for a comprehensive forensic framework that not only identifies GenAI manipulation but precisely pinpoints its location, a crucial step for actionable intelligence.

Background

▶ Watch: Introduction to GenAI image manipulation detection research (0:00)

The pervasive rise of Generative AI has ushered in a new era of digital content creation, presenting both unprecedented opportunities and significant security challenges. Matthew Thomson opened his talk by categorizing GenAI manipulation into two broad use cases: non-malicious and malicious. Non-malicious applications include creative endeavors such as movie production, photography enhancements, and general entertainment. An illustrative example provided was transforming a picture of grass into one featuring a cat running through a field—a harmless, creative modification.

However, the focus of Thomson's research, and the primary concern for digital forensics, lies in the malicious applications. These range from widespread disinformation campaigns to highly personal attacks. He cited several high-profile examples to underscore the severity:

• Political Propaganda: The widely circulated fabricated image of Volodymyr Zelenskyy and Donald Trump meeting, which, despite being debunked, demonstrated the potential for such images to incite geopolitical tensions and even conflict. Similarly, the prevalence of AI-generated images during the American election cycle highlighted their use in shaping public opinion.
• Harassment: The deeply personal threat of individuals taking someone's image from social media, manipulating it with GenAI, and using it for blackmail or reputational damage. This type of manipulation, even if the image is "not real," can cause immense psychological and social harm.
• Cybercrime: The sophisticated use of AI in phishing and social engineering attacks. A recent example involved an AI-generated impersonation of the YouTube CEO, used to defraud individuals into sending money.

Thomson's paper specifically addresses GenAI image inpainting, a technique where objects within an image are either replaced or entirely removed, seamlessly integrating the new or altered content. This differs from traditional image splicing by leveraging advanced AI models for more realistic and harder-to-detect alterations. The core problem identified in existing literature is the lack of dataset diversity. Current datasets tend to be highly specialized, focusing on narrow manipulation types like face swapping or entirely synthetic images. This specificity renders them inadequate for forensic investigations, which require tools capable of handling a broad spectrum of manipulations in diverse contexts.

For forensic practitioners, the challenge is twofold: first, identifying if an image has been manipulated by GenAI, and second, precisely localizing where that manipulation occurred. Without a diverse dataset for training and benchmarking, it becomes difficult to ascertain a tool's effectiveness across various scenarios. Furthermore, forensic analysis often lacks the luxury of an "authentic reference image"—the original, untampered version—making detection and localization significantly more complex. The research, therefore, aims to identify critical components for creating truly diverse datasets and understand how these components influence the performance of GenAI manipulation detection and localization techniques.

Key Findings

▶ Watch: Need for diverse datasets in GenAI inpainting forensics (2:37)

Matthew Thomson's research yielded several crucial insights into the landscape of GenAI image inpainting detection and localization, particularly highlighting areas where current methodologies falter and where future efforts should concentrate.

The study began by meticulously constructing a diverse dataset to overcome the limitations of existing specialized datasets. Leveraging the MS Coco dataset, Thomson selected a handful of "broader" categories. For each selected image, a prompt to "remove" an object, along with its corresponding mask, was applied using three different inpainting tools, with each process repeated four times to introduce variability. This systematic approach resulted in a comprehensive dataset containing the manipulated images, their respective masks, and the authentic reference images, providing a robust foundation for evaluation.

To assess detection capabilities, the created dataset was processed using a custom code environment incorporating multiple algorithms. This design choice was critical to ensure that the evaluation results were not biased towards a single algorithmic approach (e.g., solely noise analysis) but instead reflected a diverse range of detection techniques. The output of this processing was a probability map—a pixel-by-pixel or pixel-block representation where higher numerical values indicated a greater likelihood of tampering, according to the algorithms.

Evaluation metrics, particularly F1 scores, revealed significant disparities across different image classes. The animals and persons categories consistently showed substantially higher F1 scores compared to the objects and scenery categories. This finding suggests that the characteristics of the masked object—its shape, complexity, or perhaps typical size—play a significant role in how accurately GenAI manipulations are detected and localized. This highlights a potential area for further investigation into the impact of mask size and object complexity on detection performance.

Interestingly, the research found that the accuracy of the inpainting tools themselves did not vary significantly when evaluated within the same detection algorithm. While differences existed across various detection algorithms, the specific tool used for inpainting seemed less influential on detection success than initially hypothesized. This suggests that the underlying algorithmic approach of the detector might be more critical than the specific GenAI model used for manipulation.

Perhaps the most impactful findings came from the visual analysis of the probability maps. These visualizations, which are directly relevant to forensic practitioners, illustrated the strengths and weaknesses of current localization techniques. While many tools successfully identified the general area of the tampered object (e.g., the cow in the provided example, highlighted by yellow pixels), they frequently produced false positives in untouched, complex areas such as the sky. This tendency to misidentify pristine regions as manipulated poses a significant challenge for forensic analysts, as it can dilute the credibility of the findings and complicate the precise localization of actual tampering. The visual outputs clearly demonstrated that while an outline of the manipulated object might be discernible, the presence of widespread false positives in complex backgrounds remains a critical hurdle.

Technical Deep Dive

▶ Watch: Dataset diversity impacts detection across image categories (5:58)

Matthew Thomson's research delves into the technical intricacies of GenAI image inpainting detection, focusing on the methodology for dataset creation, the algorithmic approach to detection, and the interpretation of results. At its core, the study addresses the challenge of localization, which involves not just determining if an image has been manipulated, but precisely where the manipulation occurred.

Image Inpainting Explained:

Image inpainting is a sophisticated GenAI technique used to fill in missing parts of an image or to replace/remove existing objects seamlessly. Thomson illustrated this with a clear example: an original image of a cow in a field. A mask is applied to the cow, indicating the region to be manipulated. An inpainting tool then removes the cow, generating new pixels to fill the void, creating a manipulated image where the cow is absent. A subtle detail observed in the example was the lingering shadow of the cow, which was not part of the mask and thus remained in the "removed" image, highlighting the potential for minute forensic artifacts. The talk also briefly touched upon emerging features like AI-generated watermarks, a development spurred by regulations like the AI Act, though this was not the primary focus of the detection experiments.

Dataset Creation Methodology:

To build a robust and diverse dataset, Thomson utilized the widely recognized MS Coco (Microsoft Common Objects in Context) dataset. From this extensive collection, he strategically selected a "handful of different categories" that represented broader, more diverse scenarios than the highly specialized datasets often found in GenAI research. The process involved:

1. Category Selection: Choosing categories from MS Coco that offered a wide range of objects and scenes.
2. Prompt Application: For each selected image, a specific "remove" prompt was applied to an object identified by a mask.
3. Tool Application & Variation: The masked images and prompts were fed into three distinct inpainting tools. Crucially, this process was repeated four times for each image-tool combination, introducing variation in the generated output. This yielded a rich dataset comprising:

• The manipulated (inpainted) images.
• The precise masks used for manipulation.
• The authentic, original reference images.

Detection and Localization Framework:

The constructed dataset was then passed through a custom code framework designed for detection and localization. A key technical decision was the inclusion of multiple algorithms within this framework. This avoided reliance on a single detection paradigm (e.g., solely focusing on noise analysis or EXIF metadata analysis), ensuring a more generalized and robust evaluation of detection capabilities. The framework's objective was to assess how different components of the dataset influenced the performance of a diverse set of detection algorithms.

Evaluation and Probability Maps:

The output of the detection algorithms for each image was a probability map. These maps are essentially matrices of numbers, corresponding to individual pixels or blocks of pixels within the image. A higher number in a specific region of the probability map indicates a greater likelihood that the algorithms identified that region as tampered.

To quantify the detection accuracy, a confusion matrix approach was employed. The mask used during the inpainting process served as the ground truth reference. A true positive was recorded if the tampered result (as indicated by the probability map) was greater than or equal to the authentic result within the area defined by the mask. This rigorous comparison allowed for precise evaluation of localization accuracy.

Key Technical Observations:

• F1 Scores and Category Dependence: The analysis of F1 scores revealed a significant technical insight: detection performance was highly dependent on the category of the manipulated object. Animals and persons categories exhibited significantly higher F1 scores compared to objects and scenery. This strongly suggests that factors like the typical shape, texture, or contextual complexity associated with these categories might either introduce more detectable artifacts or present a simpler pattern for algorithms to learn. This finding warrants further investigation into variables like mask size and object complexity.
• Tool Agnosticism (within algorithms): Within a given detection algorithm, the specific inpainting tool used did not dramatically alter the detection accuracy. This implies that the fundamental artifacts left by GenAI inpainting might be similar across different tools, or that the detection algorithms are robust enough to generalize across variations introduced by different inpainting models.
• Challenges in Complex Backgrounds: The visual outputs underscored a critical technical limitation: current localization techniques frequently misidentified untouched, complex areas, particularly the sky, as tampered. This indicates that algorithms struggle to differentiate between genuine manipulation artifacts and naturally complex, high-frequency image regions, leading to false positives that hinder precise localization. This problem highlights the need for more sophisticated contextual understanding in detection algorithms.

Thomson's technical deep dive provides a clear roadmap for future research, emphasizing the need to move beyond simple detection to precise, artifact-based localization, while also addressing the challenges of real-world forensic scenarios where ground truth (authentic images) is often unavailable.

Demo / Proof of Concept

▶ Watch: Addressing forensic limitations and future research on complexity (8:07)

While Matthew Thomson's talk did not feature a live, interactive demonstration of a specific tool or proof-of-concept during the presentation, he extensively showcased the visual outputs and analytical results derived from his experimental framework. These visualizations served as compelling evidence of the framework's capabilities in detecting and localizing GenAI image inpainting.

The core of this "demonstration through results" involved presenting probability maps and their corresponding visual interpretations. Thomson displayed side-by-side comparisons of original images, the masks used for inpainting, and the outputs from various detection algorithms applied to the manipulated images. These outputs were rendered as visual overlays, using color coding to highlight detected tampered regions. For instance, yellow pixels were used to indicate areas identified as tampered, while blue pixels represented untampered regions.

A key visual example involved the image of a cow in a field. Thomson showed how different inpainting tools resulted in subtly varied manipulated images. Subsequently, the detection framework's output for these images was presented. For example, one tool's output, "Galaxy AI," effectively highlighted the region where the cow had been removed with yellow pixels, demonstrating successful localization of the actual manipulation. However, the visuals also clearly illustrated the common challenge: across all tested tools and algorithms, significant portions of the sky were also identified as tampered, despite no manipulation having occurred in those areas. This stark visual evidence underscored the issue of false positives in complex, untouched backgrounds, which is a critical limitation for forensic practitioners.

These visual results effectively functioned as a proof-of-concept for the detection and localization capabilities of the explored algorithms, while simultaneously highlighting their current limitations. They provided a tangible understanding of what a forensic analyst might "get back" from such tools, emphasizing the need for further refinement to reduce false positives and enhance the precision of localization.

Defensive Implications

▶ Watch: Developing a practical forensic framework for practitioners (11:32)

Matthew Thomson's research offers critical insights for digital forensic practitioners and security professionals seeking to defend against the growing threat of GenAI image manipulation. The implications span from immediate awareness to long-term strategic development.

For Forensic Practitioners and Analysts:

1. Awareness of Inpainting Capabilities and Limitations: Defenders must understand the sophistication of GenAI image inpainting and its potential for creating highly convincing fakes. Equally important is recognizing the current limitations of detection tools, particularly their propensity for false positives in complex, untouched areas like the sky. Relying solely on a tool's "tampered" flag without precise localization and understanding its error modes can lead to misinterpretations in investigations.
2. Focus on Localization, Not Just Detection: The research strongly emphasizes that merely knowing if an image is manipulated is insufficient. Practitioners need to know _where_ the manipulation occurred to build a credible case. Tools that provide probability maps or visual indicators of tampered regions are more valuable than those offering a binary "fake/real" assessment.
3. Challenges of Real-World Scenarios: A significant defensive implication is the stark contrast between research environments and real forensic cases. In research, authentic reference images are often available. In practice, they are not. Defenders must be aware that current research often relies on these references, and solutions for thresholding (determining manipulation without a reference) are urgently needed.
4. Consideration of Object Categories: The finding that detection accuracy varies significantly with object categories (animals/persons vs. objects/scenery) suggests that practitioners should consider the content of an image when evaluating detection results. Tools might perform better on certain types of manipulated objects, implying a need for a diverse toolkit or scenario-specific guidance.

For Developers and Researchers (Future Defensive Strategies):

1. Develop Reference-Free Detection: A primary defensive priority is the development of algorithms and tools that can accurately localize GenAI manipulation without requiring an original, authentic reference image. This will involve advanced thresholding techniques and anomaly detection that can identify subtle GenAI artifacts in isolation.
2. Address False Positives in Complex Regions: Future research and development must focus on improving the precision of localization algorithms to minimize false positives, especially in areas like skies, reflections, or intricate textures. This could involve integrating more contextual understanding or advanced artifact signature analysis.
3. Investigate Mask Size and Object Complexity: The observed correlation between detection accuracy and object categories (animals/persons versus objects/scenery) points to the importance of mask size and object complexity. Future defensive tools could be enhanced by incorporating models that are robust to varying object characteristics or by tailoring detection strategies based on the suspected nature of the manipulated content.
4. Embrace Hybrid Methods: Thomson's plan to explore hybrid methods is a crucial defensive strategy. Combining the strengths of different detection techniques—e.g., one algorithm excelling at detecting manipulations in human faces, another for environmental elements—could lead to more comprehensive and resilient defensive tools.
5. Create Practitioner-Focused Frameworks: The ultimate defensive goal is to provide practitioners with an understandable, authenticated, and practical framework. This framework should offer clear guidance on:

• Which tools to use for specific scenarios (e.g., detecting manipulated people vs. cars).
• How to use these tools effectively.
• Methods for authenticating the results for legal proceedings.
• A clear understanding of tool limitations and potential false positives.

By addressing these implications, the digital forensics community can move towards more effective and reliable methods for combating the sophisticated threats posed by GenAI image inpainting.

Key Takeaways

• GenAI image inpainting presents significant malicious potential, enabling sophisticated propaganda, harassment, and cybercrime, necessitating robust forensic detection.
• Existing datasets for GenAI manipulation detection often lack the diversity required for real-world forensic applications, hindering the development of universally applicable tools.
• Detection accuracy for GenAI inpainting varies significantly based on the category of the manipulated object, with animals and persons showing higher F1 scores than objects and scenery, suggesting the influence of mask size or object complexity.
• Current localization techniques frequently produce false positives in complex, untouched regions (e.g., skies), highlighting a critical need for improved precision and contextual understanding in detection algorithms.
• Future forensic tools must evolve to provide precise localization of manipulation without requiring an authentic reference image, a common limitation in current research.
• The path forward involves developing hybrid detection methods and a practical, practitioner-focused framework that offers clear, authenticated guidance on tool usage for diverse forensic scenarios.

About the Speaker(s)

Matthew Thomson is a doctoral researcher at Edinburgh Napier University, currently in his second year of a PhD program. His research is dedicated to the detection of GenAI manipulation in images, encompassing both fully and partially manipulated content. Thomson's work is driven by a strong desire to produce actionable insights and tools that are not only academically sound but also directly applicable and understandable for digital forensics practitioners and the broader security community. He aims to bridge the gap between complex academic research and the practical needs of those on the front lines of combating AI-driven disinformation and cybercrime.