SK-VQA: Synthetic Knowledge Generation at Scale for Training Context-Augmented Multimodal LLMs

SK-VQA: Synthetic Knowledge Generation at Scale for Training Context-Augmented Multimodal LLMs

Speakers: Xin Su, Man Luo, Kris Pan, Tien Pei Chou, Vasudev Lal, Phillip Howard

Conference: ICML 2025

YouTube: https://slideslive.com/39044012

Overview

In the rapidly evolving landscape of artificial intelligence, multimodal Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and generating content across various modalities. However, a persistent challenge remains: their struggle with knowledge-intensive tasks that demand precise, external information beyond their parametric memory. This talk introduces SK-VQA, a groundbreaking dataset designed to address this limitation by enabling the training of context-augmented multimodal LLMs. The dataset facilitates Retrieval-Augmented Generation (RAG) for multimodal models, a crucial step towards building more accurate and less hallucinatory AI systems.

Presented by Xin Su and colleagues at ICML 2025, the research highlights the critical need for high-quality, large-scale training data to unlock the full potential of RAG in multimodal settings. SK-VQA comprises over 2 million question-answer (QA) pairs, each linked to an image and a relevant context document. The talk meticulously details the synthetic generation process, rigorous filtering, and comprehensive evaluation that underpin SK-VQA's quality and diversity.

The significance of SK-VQA extends beyond mere scale; it represents a paradigm shift in how multimodal LLMs can be trained to leverage external knowledge. By providing a challenging benchmark and an effective training resource, SK-VQA empowers researchers and practitioners to develop models that can accurately answer complex, fact-based questions by reasoning over both visual and textual information. This work is pivotal for advancing the reliability and utility of multimodal AI systems in real-world applications where factual accuracy is paramount.

Background

▶ Watch: Introduction: LLMs struggle with knowledge-intensive tasks and hallucinations. (0:00)

Despite their impressive capabilities, state-of-the-art multimodal LLMs often exhibit significant limitations when confronted with knowledge-intensive visual question answering (VQA) tasks. These models, while adept at general understanding and generation, frequently struggle with specific, detailed questions about entities or events depicted in an image. For instance, an LLM might correctly identify a landmark like the Beverly Hilton, but fail to provide accurate details such as its owner or the number of suites, instead producing plausible but incorrect "hallucinated" answers based on its internal parametric memory. This suggests that the detailed, specific knowledge required for such queries is not robustly encoded within the model's parameters.

A widely recognized solution to this problem in the natural language processing (NLP) domain is context augmentation, often implemented through Retrieval-Augmented Generation (RAG) systems. In a RAG setup, an LLM is provided with additional, relevant textual context retrieved from external knowledge sources (e.g., Wikipedia) alongside the original query. This external context acts as a factual grounding, enabling the model to generate more accurate and informed responses. When applied to multimodal scenarios, this involves retrieving text passages related to an image and its associated query, then feeding both the image and the retrieved text to the multimodal LLM for answer generation.

However, a major impediment to effectively implementing RAG for multimodal LLMs has been the severe scarcity of suitable training data. Existing multimodal LLMs have largely not been trained in a context-augmented generation setting, leading to suboptimal performance when integrated into RAG systems. Unlike other common data forms (text alone or image alone), there is no naturally occurring, large-scale source of images paired with dense text passages and corresponding question-answer pairs that require reasoning over both modalities. Prior Knowledge-Based VQA (KB-VQA) datasets, such as ViQA, InfoSeek, and Encyclopedic-VQA, have been limited in scale, diversity, or relied on template-based question generation, failing to provide the rich, varied, and challenging examples needed to robustly train models for complex context-augmented reasoning. This gap in training data motivated the creation of SK-VQA.

Key Findings

▶ Watch: Introducing SK-VQA: A large-scale knowledge-based VQA dataset. (2:00)

The central contribution of this research is the introduction of SK-VQA, a novel and significantly larger Knowledge-Based Visual Question Answering (KB-VQA) dataset specifically designed to train and benchmark context-augmented multimodal LLMs. The key findings and contributions are multifaceted:

1. Unprecedented Scale and Diversity: SK-VQA comprises over 2 million QA pairs associated with 290,000 unique images and context documents. This represents an order of magnitude increase in scale and diversity compared to prior KB-VQA datasets. The dataset features a much broader range of topics (e.g., art, fashion, sports, music) and more diverse, natural-sounding questions, generated using GPT-4, unlike the often template-based questions in older datasets.

1. Effective Synthetic Data Generation Pipeline: The research demonstrates a robust methodology for synthetically generating high-quality context documents and QA pairs at scale. By leveraging GPT-4 to simultaneously generate context and associated QA pairs, the pipeline produces information-dense passages well-suited for training models in context-augmented reasoning. Rigorous filtering mechanisms, including an image reference filter and a context-answer presence filter, ensure the quality and utility of the generated data.

1. Challenging Benchmark for Multimodal LLMs: When evaluated in a zero-shot setting, SK-VQA proved to be one of the most challenging benchmarks for popular open-source multimodal LLMs, performing roughly on par with InfoSeek and sometimes even exceeding its difficulty. This confirms SK-VQA's utility for assessing the context-augmented reasoning capabilities of these models.

1. Superior Training Resource for Out-of-Domain Generalization: Fine-tuning experiments with models like LLaVA and PaliGemma revealed that training on SK-VQA consistently led to significant improvements in out-of-domain generalization performance across other KB-VQA datasets. In contrast, training on other existing KB-VQA datasets often resulted in performance degradation on out-of-domain tasks, highlighting SK-VQA's unique effectiveness as a training resource.

1. Validation of High-Quality Synthetic Data: A surprising and notable finding was that training on fully synthetic image and context documents from SK-VQA yielded performance that was as good as, or even slightly better than, training on real data sourced from Wikipedia. This challenges the common assumption in prior studies that synthetic data is inherently less efficient or effective than real data, opening new avenues for large-scale data generation.

1. Comprehensive Quality Assurance: The dataset underwent extensive automated and human evaluations, confirming its high quality. Automated checks using GPT-4o as a judge yielded high scores for factuality (4.6/5), question relevancy (4.9/5), answerability (~100%), and answer correctness (~91%). Human annotators achieved approximately 87% human-level accuracy on the most filtered subset, validating the dataset's reliability.

Technical Deep Dive

▶ Watch: SK-VQA dataset creation using GPT-4 for synthetic generation. (3:00)

The technical foundation of SK-VQA lies in its sophisticated data generation pipeline, meticulous filtering, and rigorous evaluation methodologies. The goal was to create a dataset where answers to questions require joint reasoning over both an image and a provided text context, with the answer explicitly present within the context.

The dataset creation process began with acquiring images from diverse sources to ensure broad coverage. These sources included the LAION dataset, general Wikipedia images, and the synthetic COCO-Counterfactuals dataset. This multi-source approach allowed for a wider range of visual content compared to prior datasets that often relied solely on images already associated with text (like Wikipedia).

For each acquired image, a corresponding context document was generated. The vast majority of these context documents were synthetically produced using GPT-4. A smaller portion of real context documents was procured directly from Wikipedia for cases where Wikipedia images could be directly associated with their existing text. A critical aspect of this generation strategy was that GPT-4 was prompted to generate both the context document and multiple associated QA pairs in the same inference pass. This co-generation process incentivized GPT-4 to produce context passages that were highly information-dense and directly relevant to the subsequent task of answering questions, making them more suitable for context-augmented reasoning than general natural text.

To ensure the high quality and specific utility of SK-VQA for context-augmented training, two primary filtering mechanisms were implemented:

1. Image Reference Filter: This filter was designed to remove context documents that directly referenced the input image using phrases like "in this image" or "as shown in this picture." Such references often indicate that the text is more akin to an extended caption rather than an independent, information-dense passage requiring multimodal integration. By removing these, the dataset encourages models to learn to interpret the image and context as distinct but complementary sources of information. This filter led to the creation of the SK-VQA-IR subset.

1. Context-Answer Presence Filter: This is a standard filter in KB-VQA datasets, ensuring that the answer to every question is explicitly present within the provided context document. This condition is crucial for training models to effectively extract information from the context. It also guarantees that the context document is indeed necessary to answer the question, validating the core premise of context augmentation. Applying this filter alongside the image reference filter created the most stringently filtered subset, SK-VQA-IRCAC, which contains approximately half the total QA pairs of the full dataset.

The diversity of SK-VQA was extensively analyzed and compared against existing KB-VQA datasets like ViQA, InfoSeek, and Encyclopedic-VQA. The analysis revealed that SK-VQA not only contains nearly 50% more total questions but also boasts an order of magnitude more unique questions, significantly greater unique parts of speech, and a larger vocabulary size. This superior linguistic diversity is attributed to the use of GPT-4 for natural-sounding question generation, contrasting with the often template-based methods of prior efforts. Furthermore, a topic model fitted to the context documents demonstrated a much broader range of topics in SK-VQA, including categories such as art, fashion, sports, events, and music, moving beyond the predominantly entity-specific knowledge found in older datasets.

Experimental Setup & Results

▶ Watch: SK-VQA size and unparalleled diversity compared to prior datasets. (5:00)

The research conducted a series of comprehensive experiments to evaluate SK-VQA's utility as both a benchmark and a training resource.

Benchmark Evaluation (Zero-Shot):

Multimodal LLMs were evaluated in a zero-shot setting on SK-VQA, as well as on ViQA, InfoSeek, and Encyclopedic-VQA. While specific model names were mentioned for fine-tuning, the talk indicated results for popular open-source models in the main presentation and commercial models in the appendix. The key finding here was that SK-VQA consistently proved to be among the most challenging datasets for these models, often on par with InfoSeek and sometimes even more difficult. This established SK-VQA as a robust benchmark for assessing context-augmented reasoning.

Fine-tuning Experiments:

To evaluate SK-VQA as a training resource, variants of LLaVA (a popular open-source multimodal LLM) and PaliGemma were fine-tuned.

• Training Datasets: Models were fine-tuned on SK-VQA (including its filtered subsets SK-VQA-IR and SK-VQA-IRCAC), InfoSeek, and Encyclopedic-VQA. For a fair comparison, SK-VQA was downsampled to an equivalent amount of data as obtainable from the other sources.
• Evaluation Setting: Each fine-tuned model was then evaluated on all the KB-VQA datasets (including those it was not trained on) to assess its out-of-domain generalization performance. Experiments were conducted in two settings: utilizing gold context documents (the actual documents associated with each QA pair) and retrieved context documents (mimicking a RAG system). The talk primarily showed results for gold context, noting that trends were very similar for retrieved context.
• Results: The experiments revealed a striking trend: only by training on SK-VQA (and its filtered subsets) did models consistently show improvements over the baseline (untrained) model in out-of-domain generalization. Training on other datasets (InfoSeek, Encyclopedic-VQA) often led to performance degradation, particularly on datasets like ViQA and Encyclopedic-VQA. This strongly suggests that SK-VQA provides the necessary scale and diversity to effectively teach multimodal LLMs how to leverage external context for generalized reasoning.

Impact of Image and Context Sources (Synthetic vs. Real Data):

A specific ablation study was conducted for LLaVA to explore the impact of different image and context sources. Models were fine-tuned on subsets of SK-VQA sampled exclusively from a single image and context source.

• Comparison: The critical comparison was between training on real images and context documents sourced from Wikipedia versus training on fully synthetic image and context documents (from the LAION/COCO-Counterfactuals + GPT-4 pipeline).
• Results: Surprisingly, training on fully synthetic image and context documents was found to be as good as, or even slightly better than, training on real data from Wikipedia. This is a significant finding that challenges previous observations in the literature, where synthetic data is often considered less efficient than real data. It underscores the high quality and effectiveness of the SK-VQA synthetic data generation pipeline.

Quality Evaluation:

Given the synthetic nature of much of the dataset, rigorous quality evaluations were performed:

• Automated Evaluations:
• Grammatical Correctness: Less than 7% of the text was flagged for issues, with over 80% of these being minor spelling or punctuation problems, not impacting utility.
• Bias and Toxicity: Standard classifiers found no instances of bias or toxic content.
• GPT-4o as Judge: GPT-4o was used to assess four dimensions on a 1-5 scale:
• Factuality of context documents: Average score of 4.6 out of 5.
• Question relevancy: Average score of 4.9 out of 5.
• Answerability: Nearly 100%.
• Answer correctness: Around 91%.
• Human Evaluations:
• Setup: 100 QA pairs from the most filtered subset (SK-VQA-IRCAC) were sampled, and three human annotators assessed data quality and attempted to answer questions using only the image and context.
• Results: Humans achieved approximately 87% accuracy on the most filtered subset, which is on par with other similar datasets.
• Identified Deficiencies: Annotators identified minor issues: about 9% of questions could be answered solely from the context (without the image), 5% solely from the image (without context), and 1% could not be answered due to insufficient information. The presenter noted that most of these cases were not significant, as they still often required reasoning over context documents or initial retrieval based on the image.

Practical Implications

▶ Watch: Evaluating multimodal LLMs using SK-VQA as a benchmark. (6:50)

The introduction of SK-VQA carries significant practical implications for practitioners, infrastructure teams, model builders, and deployers working with multimodal AI systems.

Firstly, SK-VQA directly enables the development of more robust and reliable Multimodal LLMs (MLLMs) for Retrieval-Augmented Generation (RAG). The lack of suitable training data has been a major bottleneck for integrating external knowledge into MLLMs. SK-VQA fills this gap, providing the necessary scale and quality to fine-tune models to effectively leverage retrieved context for knowledge-intensive VQA tasks. This means future MLLMs can be deployed in applications where factual accuracy is paramount, such as scientific research, journalism, or customer support, significantly reducing the incidence of hallucinations.

Secondly, the research provides a strong validation for the efficacy of high-quality synthetic data generation. The finding that fully synthetic data can be as effective as, or even superior to, real data for training challenges established paradigms. This has profound implications for data scarcity issues in AI. For domains where real, human-annotated multimodal data is expensive, rare, or privacy-sensitive, synthetic generation pipelines like the one used for SK-VQA offer a scalable and cost-effective alternative. Infrastructure teams can invest in robust generative AI models (like GPT-4) to create specialized datasets rather than relying solely on manual annotation efforts.

Thirdly, SK-VQA serves as a powerful and challenging benchmark for evaluating the context-augmented reasoning capabilities of MLLMs. Practitioners can use SK-VQA to rigorously test and compare different model architectures, RAG strategies, and training methodologies. Its diversity in question types and topics ensures that models are assessed on a broad range of knowledge, pushing the boundaries of what MLLMs can achieve in complex, real-world scenarios.

However, there are also practical tradeoffs and limitations to consider. The context-answer presence filter, while vital for training models to extract information, means that the dataset inherently assumes the answer exists within the provided context. In real-world RAG systems, users might ask questions for which no answer is available in the retrieved documents. While the speaker suggested that models trained on SK-VQA would likely still be able to indicate when an answer is not present, this specific capability is not directly trained by the current dataset design. Future work might explore training paradigms that explicitly teach models to identify and gracefully handle unanswerable questions given the context. Additionally, while document length did not show significant performance degradation in experiments, the optimal length and density of retrieved context for different MLLM architectures remains an area for further empirical investigation.

Key Takeaways

• SK-VQA is the largest and most diverse KB-VQA dataset to date, offering over 2 million QA pairs with associated images and context documents, addressing a critical data scarcity for multimodal RAG.
• High-quality synthetic data generation using GPT-4 is highly effective, producing information-dense context documents and diverse QA pairs that enable robust training of context-augmented MLLMs.
• SK-VQA serves as a challenging benchmark, pushing the limits of current multimodal LLMs in zero-shot context-augmented reasoning tasks.
• Training on SK-VQA consistently improves out-of-domain generalization for MLLMs like LLaVA and PaliGemma, unlike other existing KB-VQA datasets which often lead to performance degradation.
• Synthetic data can be as effective as real data for training MLLMs, challenging prior assumptions and opening new avenues for scalable and cost-efficient data generation.
• Rigorous filtering and evaluation (automated and human) ensure SK-VQA's high quality, with strong scores for factuality, relevancy, answerability, and correctness.

About the Speaker(s)

The talk on SK-VQA was presented by Xin Su, who was joined by collaborators Man Luo, Kris Pan, Tien Pei Chou, Vasudev Lal, and Phillip Howard. While specific titles and affiliations were not detailed in the transcript, their collective work presented at ICML 2025 indicates a strong background in AI/ML research, particularly in the areas of multimodal learning, large language models, and data generation for complex AI systems. The presentation highlighted their expertise in developing novel datasets and methodologies to address fundamental challenges in making AI models more knowledgeable and reliable. Xin Su also mentioned opportunities for PhD students seeking research internships, suggesting an affiliation with a research-intensive organization.