Generative Social Choice: The Next Generation

Generative Social Choice: The Next Generation

Speakers: Niclas Boehmer, Sara Fish, Ariel Procaccia

Conference: ICML 2025

YouTube: https://slideslive.com/39044108

Overview

In an era increasingly shaped by digital discourse and the proliferation of diverse viewpoints, the challenge of accurately and proportionally summarizing collective opinion is paramount. This talk, "Generative Social Choice: The Next Generation," presented by Sara Fish, Niclas Boehmer, and Ariel Procaccia, addresses this complex problem by introducing a novel framework for proportional summarization of large, unstructured datasets of user opinions. The core objective is to generate a concise "slate" of statements that reflect the varying support levels across a user population, ensuring that an X fraction of users "control" an X fraction of the output. This has profound implications for applications in AI and democracy, where technologies are leveraged to enhance collective decision-making processes.

The central dilemma tackled by the researchers lies in the inherent tension between the powerful capabilities of Large Language Models (LLMs) and their known limitations regarding trustworthiness, such as hallucination and lack of explainability, particularly in high-stakes contexts. While LLMs offer unprecedented flexibility for processing natural language inputs and generating sophisticated outputs, their integration into systems designed for democratic participation necessitates robust guarantees. The presented work, building upon prior research in Generative Social Choice, provides a principled approach to harness LLM strengths while mitigating these risks, thereby advancing the field of AI alignment with guarantees.

This next-generation approach significantly enhances the robustness of theoretical guarantees, moving beyond assumptions of perfect LLM performance to account for realistic error parameters in the underlying query mechanisms. By empirically evaluating these queries, the framework transforms the daunting task of establishing trust in an entire LLM-based system into the more manageable problem of verifying specific, well-defined black-box operations. The practical implementation, dubbed PROS, leverages GPT-4o and demonstrates superior performance in generating proportional summaries compared to existing baselines, promising more equitable and representative outcomes in digital civic engagement and beyond.

Background

▶ Watch: Introduction to proportional summarization and its goal (0:00)

The motivation for this research stems from the growing need for sophisticated tools to understand and aggregate public opinion, particularly within the context of AI and democracy. Platforms like Polis exemplify successful systems that analyze user votes on statements to map public sentiment, cluster users by voting behavior, and facilitate collective decision-making. However, the advent of Large Language Models (LLMs) presents both a tremendous opportunity and a significant challenge to such systems. LLMs, with their ability to process and generate natural language, can offer unparalleled flexibility in handling diverse inputs and outputs, potentially revolutionizing how opinions are gathered and summarized.

Despite their power, integrating LLMs into sensitive, high-stakes environments like civic participation platforms immediately raises critical concerns. Issues such as hallucination (generating factually incorrect or unsupported information), a lack of explainability (difficulty in understanding why an LLM made a particular decision), and general untrustworthiness become paramount. These concerns necessitate a framework that can leverage LLMs' generative capabilities without compromising the integrity and fairness of the overall decision-making process.

To address these conflicting objectives, the presented work adopts and extends a framework from prior research, Generative Social Choice, on which some of the current speakers are also authors. This framework is part of a broader, emerging literature focused on AI alignment with guarantees, aiming to ensure that AI systems operate in a predictable, fair, and trustworthy manner. The core idea is to decouple the potentially untrustworthy LLM components from the rest of the system as much as possible. This is achieved by specifying the overall process in terms of abstract, black-box queries. Theoretical guarantees about the system's behavior (e.g., proportionality) are then proven based on assumptions about these queries. Subsequently, LLMs are used to implement these queries, and their trustworthiness is established through empirical evaluation. This modular approach simplifies the problem: instead of trying to trust a complex, end-to-end LLM-based system, one only needs to establish trust in the performance of a few well-defined queries.

The specific problem of proportional summarization is reframed within social choice theory, a field dedicated to aggregating individual preferences into collective decisions. This allows the researchers to draw upon established concepts and mechanisms, particularly from participatory budgeting, a domain where a group decides how to allocate resources or funds. In this context, users are considered "voters" with preferences over all possible statements (candidates), and the goal is to select a subset of these statements (a "slate") that respects a total length budget while ensuring proportionality. The key innovation lies in navigating the astronomically large candidate space—all conceivable natural language statements—by employing LLM-powered queries, a task unfeasible with traditional social choice methods.

Key Findings

▶ Watch: Key challenge: Trustworthy LLM integration in high-stakes systems (1:50)

The primary contribution of this research lies in significantly enhancing the robustness of theoretical guarantees for proportional summarization, particularly in the presence of imperfect LLM outputs. While prior work in Generative Social Choice established guarantees under the idealized assumption of "perfect queries," this paper introduces and proves approximate proportionality guarantees that explicitly account for query errors. This means the theoretical strength of the guarantees is now proportional to the extent of these error parameters, offering a more realistic and applicable framework for real-world LLM deployments.

The researchers developed and implemented an iterative greedy algorithm, PROS, which stands for PROportional Summarization, to construct the output slate. This algorithm strategically leverages two types of LLM-powered black-box queries—a generative query for creating statements and a discriminative query for evaluating user agreement—to navigate the vast space of potential statements and ensure proportional representation. The theoretical underpinnings of PROS demonstrate that if these queries are answered perfectly, the algorithm yields perfect proportionality. Crucially, even with noisy query answers, PROS provides provably approximate proportionality guarantees, which are shown to be close to optimal given the inherent noise.

In empirical evaluations, PROS, implemented using GPT-4o, demonstrated superior performance compared to several LLM-based zero-shot baselines and a baseline resembling the original Generative Social Choice approach. Specifically, PROS achieved significantly fewer proportionality violations, indicating a more equitable representation of diverse user opinions in the generated summaries. Furthermore, PROS consistently yielded higher mean average agent utility, signifying that users generally preferred the statements representing them in the output slate. Notably, the algorithm also achieved a higher lowest quartile utility, underscoring its success in ensuring that even marginalized and smaller minority groups were adequately represented, thereby fostering more inclusive collective decision-making. These findings collectively validate the framework's ability to combine the expressive power of LLMs with rigorous social choice principles to achieve trustworthy and proportional summarization.

Technical Deep Dive

▶ Watch: Main contribution: Robust theoretical guarantees with error parameters (4:00)

The technical core of "Generative Social Choice: The Next Generation" revolves around framing the proportional summarization task as a social choice problem and developing an iterative algorithm that relies on carefully designed black-box queries implemented by LLMs. The input to the system is unstructured user data in natural language, coupled with a budget B, which specifies the maximum length of the output summary "slate" in words.

The challenge in interpreting this as a social choice problem is the immense "candidate space"—all possible statements or sentences that could be included in the summary. Traditional social choice mechanisms are not designed to handle such a vast and dynamically generated set of candidates. To address this, the framework introduces two critical query types:

1. Generative Query: This query takes as input a subset of users, an "approval level" (indicating the desired degree of agreement), and a length constraint. Its assumed function is to produce a high-quality statement that is liked by a significant portion of the input users at the specified approval level, while also respecting the given length limit. This query effectively prunes the infinite candidate space by generating relevant and representative statements on demand.
2. Discriminative Query: For a given single user and a specific statement, this query outputs a rating that quantifies how much that user agrees with or "likes" the statement. This provides the necessary feedback mechanism to evaluate the quality and representativeness of generated statements across the entire user population.

The Iterative Greedy Slate Generation Algorithm then constructs the proportional summary:

• Initialization: The algorithm starts with an empty slate and considers the entire set of users.
• Iterative Statement Addition: Statements are added one by one to the slate. After a statement is added, the users who are deemed "covered" by that statement are removed from the active user set, ensuring that each user is eventually represented.
• Exploration of Options: In each iteration, the algorithm explores various combinations of "approval levels" and "budget levels" for the next statement. These parameters guide how much the next statement should be liked by users and its target length, respectively.
• Statement Generation and Evaluation: For each such combination, the generative query is invoked to produce a batch of potential statements. These statements are then evaluated against all remaining users using the discriminative query to ascertain each user's agreement.
• Selection and Proportionality Check: The algorithm identifies the statement that is liked by the most users at the current approval level. Before adding it to the slate, a crucial proportionality check is performed. This check determines if the statement is sufficiently backed by users to justify its length, adhering to the principle that "every user deserves to be represented by B/N many words" (where N is the total number of users and B is the total budget). Specifically, the condition checked is: (number of users who like the current statement) * (B / N) > (length of the next statement to be added).
• Slate Update: If the proportionality condition is met, the statement is added to the slate, and the users who like this statement are marked as covered and removed from the active user set.
• Termination: This process repeats until all users are covered by at least one statement in the slate.

A key theoretical contribution is the proof of approximate proportionality guarantees. While perfect queries lead to perfect proportionality, the paper rigorously shows that the algorithm provides guarantees parameterized by the extent of errors in the query answers. This robustness is crucial for practical applications where LLMs are inherently imperfect. Furthermore, the research includes impossibility results, demonstrating that no algorithm can achieve significantly better proportionality guarantees given only access to noisy query answers, implying the presented algorithm is near-optimal under such conditions.

The practical implementation, PROS, leverages GPT-4o for both generative and discriminative queries:

• Discriminative Query Implementation: This is achieved through two fast LLM calls. These calls assess different facets of user agreement with a given statement, and their outputs are then combined into a final agreement rating. The use of "fast" calls suggests an emphasis on efficiency and potentially prompt engineering to elicit specific, quantifiable agreement signals.
• Generative Query Implementation: This is a two-step process. First, to ensure statements are relevant to cohesive groups, a clustering algorithm is applied to user data in a customized embedding space. This identifies groups of users with similar opinions. Second, for each identified cohesive user group, GPT-4o is prompted to generate a statement that is supported by all users within that specific group, ensuring targeted and representative content generation.

This detailed technical design, from the social choice framing to the LLM-powered query implementation and the iterative algorithm with provable guarantees, forms a comprehensive solution to proportional summarization that balances the power of modern AI with the need for fairness and trustworthiness.

Experimental Setup & Results

▶ Watch: Framing proportional summarization as a social choice problem (4:50)

To validate the theoretical framework and the practical implementation of PROS, the researchers conducted extensive experiments using diverse datasets and compared their approach against several baselines. The experimental setup focused on evaluating the algorithm's ability to generate proportional summaries and the quality of representation for individual users.

Datasets:

The evaluation utilized two distinct types of data:

1. Web-scraped drug reviews: This dataset provided a rich source of unstructured natural language opinions, representing a broad spectrum of individual experiences and sentiments related to specific products. This kind of data is challenging due to its inherent variability and potential for subjective expression.
2. Polis Bowling Green dataset: This real-world dataset originated from the Polis platform, a system designed for collective decision-making. In this specific instance, citizens from Bowling Green discussed various ways to improve their city. This dataset offers a compelling testbed for the framework's application in civic participation contexts, reflecting genuine public discourse and diverse perspectives on policy-relevant issues.

Baselines:

The performance of PROS was benchmarked against four baselines:

• Two LLM-based zero-shot baselines: These likely represented straightforward applications of LLMs to the summarization task without the iterative, query-driven structure of PROS. Such baselines often lack explicit mechanisms for ensuring proportionality.
• A baseline resembling the original Generative Social Choice approach: This baseline likely implemented the prior framework's query mechanism, but potentially without the enhanced robustness guarantees for query errors or the specific two-step generative and discriminative query implementations developed for PROS.

Metrics:

The evaluation primarily focused on two key metrics:

1. Proportionality Violations: This was the central measure of fairness and representativeness. A proportionality violation was defined when a sufficiently large number of users preferred a newly generated statement over the statement by which they were currently represented in the output slate, and these users were numerous enough to "afford" this new statement (i.e., their collective "budget share" was greater than the statement's length). Fewer violations indicate a more proportionally representative summary.
2. Agent Utility: This metric quantified how much individual users "liked" or agreed with the statements chosen to represent them in the final slate. It was measured in terms of both mean average agent utility (overall satisfaction) and lowest quartile utility (satisfaction of the least satisfied 25% of users), with the latter being particularly important for assessing the representation of minority or marginalized viewpoints.

Headline Results:

The experimental results unequivocally demonstrated the superior performance of PROS:

• Significantly Fewer Proportionality Violations: PROS achieved substantially fewer proportionality violations compared to all four baselines. This is a critical finding, directly supporting the framework's core objective of generating more equitable and representative summaries. While some violations still occurred, indicating that "certainly not all queries are answered perfectly," the reduction was significant, highlighting the effectiveness of the iterative algorithm and robust query design.
• Higher Mean Average Agent Utility: Users represented by the PROS-generated slates reported higher overall satisfaction with their assigned statements. This indicates that the algorithm not only achieved proportional representation but also did so with statements that genuinely resonated with the users they aimed to cover.
• Higher Lowest Quartile Utility: Crucially, PROS also exhibited a higher lowest quartile utility. This particular result is highly significant as it confirms the algorithm's success in representing marginalized and smaller minority groups. In democratic processes, ensuring that the voices of less dominant factions are heard and reflected is paramount, and PROS demonstrated a clear advantage in this regard.

These results collectively underscore the practical efficacy of the "Generative Social Choice: The Next Generation" framework, showcasing its ability to produce high-quality, proportionally representative summaries even with real-world, noisy LLM outputs.

Practical Implications

▶ Watch: Iterative greedy slate generation algorithm overview (6:50)

The "Generative Social Choice: The Next Generation" framework offers several profound practical implications for practitioners, infrastructure teams, model builders, and deployers working with AI/ML systems, particularly in sensitive domains.

Firstly, the most significant implication is the increased trustworthiness of LLM-based algorithms in high-stakes decision-making contexts. By adopting a query-based framework that decouples the opaque LLM components from the core decision-making logic, the research provides a blueprint for building more accountable AI systems. Instead of grappling with the inherent black-box nature of an entire LLM, practitioners can focus on empirically evaluating and establishing trust in a few, well-defined black-box queries. This transforms the complex problem of LLM trustworthiness into a more manageable one of query validation, making LLM deployment in critical applications like civic tech more viable and less risky.

Secondly, the framework directly enables new and exciting forms of civic participation. Platforms like Polis, which aim to aggregate and understand public opinion, can significantly benefit. By using LLMs to generate and evaluate statements in a proportionally guaranteed manner, these platforms can create more representative summaries of public discourse, ensuring that diverse viewpoints, including those of minorities, are accurately reflected. This could lead to more inclusive policy-making, better community engagement, and a more robust digital democracy. Infrastructure teams can design systems around this modular query architecture, allowing for easier integration and updates of LLMs without re-evaluating the entire system's theoretical guarantees.

Thirdly, the introduction of approximate proportionality guarantees parameterized by query errors is a crucial advancement for model builders. It acknowledges the inherent imperfection of LLMs and provides a theoretical safety net. This means that even if an LLM occasionally hallucinates or misinterprets, the overall system can still provide a bounded level of proportionality. This robustness allows for more practical deployments, where perfect LLM performance cannot be assumed. Deployers can use these error parameters to quantify the risk and confidence level associated with their LLM-powered summarization systems.

However, there are also practical tradeoffs and limitations. The reliance on LLMs, even for specific queries, means that the quality and biases of the underlying LLM (e.g., GPT-4o) will still influence the system's output. While the framework provides theoretical bounds on the impact of errors, the ultimate fairness and quality of the generated statements are tied to the LLM's performance. Prompt engineering, fine-tuning, and continuous monitoring of the LLM queries remain essential tasks for model builders and infra teams. Furthermore, the computational cost of running numerous LLM calls (especially the generative query for multiple approval/budget levels and the discriminative query across all users) could be a practical concern, particularly for very large user populations or strict latency requirements. The identification of "cohesive user groups" via clustering also introduces another layer of potential bias or error that needs careful consideration.

In summary, this work provides a powerful paradigm for integrating LLMs into sensitive applications by offering a principled way to manage trustworthiness and ensure fairness. It empowers practitioners to build more robust, transparent, and democratic AI systems, albeit with the ongoing need for careful LLM management and resource optimization.

Key Takeaways

• Trustworthy LLM Integration: The framework decouples untrustworthy LLM components from the core decision-making process, enabling the use of powerful LLMs in high-stakes systems (like civic participation) while maintaining trustworthiness.
• Robust Proportionality Guarantees: Moving beyond idealized assumptions, the work provides approximate proportionality guarantees that account for realistic LLM query errors, enhancing the practical applicability and reliability of the system.
• Iterative Query-Based Algorithm: The PROS algorithm leverages generative and discriminative queries (implemented with GPT-4o) in an iterative greedy fashion to construct proportionally representative summaries from unstructured natural language data.
• Superior Performance: Empirical results demonstrate that PROS significantly reduces proportionality violations and achieves higher agent utility (including for marginalized groups) compared to zero-shot LLM baselines and prior Generative Social Choice approaches.
• Enabling Civic Participation: By ensuring fair and proportional representation of diverse opinions, this research offers a pathway for developing more equitable and inclusive AI tools for collective decision-making and digital democracy.
• Problem Transformation: The framework transforms the complex challenge of trusting an entire LLM-based system into the simpler, more manageable task of empirically evaluating the quality and consistency of specific black-box queries.

About the Speaker(s)

Sara Fish is a fourth-year PhD student and a key collaborator on this research. She played a significant role in presenting the work and is an author on the prior Generative Social Choice paper, highlighting her ongoing contributions to this field.

Niclas Boehmer is a collaborator on this research and co-presented the talk with Sara Fish. His expertise is central to the theoretical and algorithmic contributions of the "Generative Social Choice: The Next Generation" project.

Ariel Procaccia is a prominent figure in the field and an author on the original Generative Social Choice work. His involvement underscores the continuity and evolution of this research agenda, particularly in the intersection of AI, social choice, and democratic processes.