A Generalization Theory for Zero-Shot Prediction

Ronak Mehta, Zaid Harchaoui

International Conference on Machine Learning 2025 · Oral

In this insightful talk from ICML 2025, Ronak Mehta, in collaboration with his advisor Zaid Harchaoui, presents a groundbreaking theoretical framework for understanding the generalization capabilities of **zero-shot prediction (ZSP)**. Prompted by the rapid advancements and perplexing efficacy of **foundation models**—encompassing large language models and multimodal embedding models—the research addresses a critical gap in the theoretical understanding of their practical applications. While these models have revolutionized tasks like image classification without direct labeled training data, a robust theoretical analysis akin to classical statistical learning theory has been conspicuously absent.

AI review

Mehta and Harchaoui take a genuinely important and undertheorized problem — the generalization behavior of zero-shot prediction via foundation models — and provide a principled decomposition of the error into an information-theoretic component and a statistical learning component. The four-quantity framework (conditional dependence, prompt bias, sample complexity, prompt complexity) is the kind of clean, actionable organizing structure that a subfield can actually build on. The work earns serious credit for formalizing something the community has been handwaving about for years. My…