CollabLLM: From Passive Responders to Active Collaborators

Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, Jianfeng Gao

International Conference on Machine Learning 2025 · Oral

In an era where **Large Language Models (LLMs)** are increasingly integrated into daily workflows, from drafting documents to solving complex scientific problems, the quality of human-LLM collaboration has become paramount. This talk introduces **CollabLLM**, a groundbreaking framework designed to transform LLMs from passive, reactive systems into active, collaborative partners. Presented by Shirley Wu at ICML 2025, CollabLLM addresses a critical limitation of current LLMs: their tendency to jump to premature conclusions and fail to proactively seek clarifying information, leading to inefficient and often frustrating interactions.

AI review

CollabLLM proposes a multi-turn reward mechanism for training LLMs to ask clarifying questions by rolling out synthetic future conversations via an LLM-based user simulator and averaging the resulting task/efficiency/interactivity scores as a training signal. The empirical results are reasonably convincing — a 200-person human study is a genuine effort — and the problem framing is honest about a real failure mode in RLHF-trained systems. But the theoretical underpinning is thin, the 'causal effect' framing is asserted rather than formalized, and the central technical idea is a fairly direct…