Position: AI Safety should prioritize the Future of Work

Position: AI Safety should prioritize the Future of Work

Speakers: Sanchaita Hazra, Bodhisattwa Prasad Majumder, Tuhin Chakrabarty

Conference: ICML 2025

YouTube: https://slideslive.com/39043982

Overview

This talk, presented by Tuhin Chakrabarty on behalf of himself and co-authors Sanchaita Hazra and Bodhisattwa Prasad Majumder, delivers a compelling position statement arguing for a fundamental reorientation of the AI safety paradigm. Moving beyond the often-polarized debates around existential risks versus unbridled innovation, the speakers assert that AI safety must prioritize the immediate and tangible impacts of AI on the future of work, human labor, and societal equity. The core argument posits that current AI research practices and governance frameworks are inadequately addressing critical issues such as widespread job displacement, skill disparity, cognitive debt, and the erosion of creative industries, thereby jeopardizing meaningful human labor.

The presentation critically examines how the rapid proliferation of advanced AI, particularly generative AI (GenAI) and agentic AI, is creating significant technical debt and fostering impractical automation across various sectors. It highlights the direct correlation between AI adoption and the decline of jobs in fields like software development and writing, while also exposing the detrimental effects of AI over-reliance on learning and knowledge creation. The speakers advocate for a systematic overhaul of AI governance, emphasizing pro-worker frameworks, robust social safety nets, and a reduction in the dominance of big tech monopolies, ultimately aiming to build AI that genuinely serves humanity rather than displacing it.

This position paper is particularly significant as it challenges the prevailing narratives within the AI safety community, which often focus on hypothetical long-term catastrophic scenarios. Instead, it grounds the discourse in observable, current-day harms and structural inequities perpetuated by unchecked AI development. By spotlighting issues like copyright infringement, data colonization, and the imperative for ethical attribution, the talk provides a timely and practical framework for ensuring that AI's transformative potential is harnessed responsibly, fostering shared prosperity rather than exacerbating economic and social divides.

Background

▶ Watch: Introduction and position on AI safety (0:00)

The field of AI safety has experienced an explosive growth in recent years, yet it remains deeply polarized, characterized by a fundamental dichotomy in perspectives. One dominant school of thought within the AI safety community largely focuses on existential risks (x-risk) and catastrophic risks, positing that advanced AI could pose an ultimate threat to humanity's survival. This perspective often drives discussions around "runaway AI" or scenarios where AI agents autonomously develop capabilities that lead to unintended and irreversible harm on a global scale. In contrast, another prevalent view emphasizes AI's immense potential for innovation, productivity gains, and problem-solving, often downplaying or debunking claims of widespread job displacement or inherent unsafety, framing AI as a tool for human augmentation.

The speakers argue that both these extreme positions often overlook the more immediate, pervasive, and structurally embedded challenges that AI, particularly foundation models and generative AI, introduces into the socio-economic fabric. The problem is not merely a distant hypothetical threat, nor is it a universally beneficial technological wave; rather, it is a complex issue rooted in the current development and deployment practices of AI systems. A significant concern raised is the accumulation of technical debt and the push towards impractical automation. The rapid "AI arms race" has led to an explosion of AI agents and automated systems that, while promising efficiency, often disrupt stable income cycles, accelerate skill disparity, and diminish worker bargaining power and economic security.

Empirical observations underscore this growing problem. The talk cites data from Anthropic's API economic index, indicating that a staggering 50% of AI usage primarily concentrates on software development and writing tasks. Concurrently, a paper from the Harvard Business School revealed a steep decline in these very job categories, establishing a direct correlation between AI adoption and labor market shifts. This confluence of factors creates a pressing need to address the "future of work" within the AI safety discourse, arguing that the consequences on human labor, learning, and economic justice are not secondary concerns but central to ensuring that AI truly benefits humanity. The existing problem, therefore, lies in a disconnect between the abstract, long-term focus of much AI safety research and the tangible, near-term socio-economic disruptions already underway due to unchecked AI development.

Key Findings

▶ Watch: Problem 1: AI automation, technical debt, job disruption (2:00)

The talk presents several critical findings that underscore the urgent need to prioritize the future of work within the AI safety agenda:

1. Technical Debt and Impractical Automation: The rapid proliferation of agentic AI and AI agents is not only leading to significant technical debt but also fostering impractical automation. This trend disrupts consumption smoothing by eliminating stable income cycles, accelerates skill disparity by outpacing society's ability to adapt, and diminishes worker bargaining power and economic security. The speakers illustrate this with data showing a steep decline in automation-prone jobs compared to manual-intensive ones.

1. Generative AI Impairs Learning and Knowledge Creation: The widespread use of generative AI (GenAI), particularly in academic and educational settings, is shown to significantly impair learning. Students relying on tools like GPT-4 as a "crutch" for tasks like math problems perform worse when AI assistance is removed. More than 50% of AI usage in education is for direct problem-solving, as per Anthropic's education index, leading to what the MIT Media Lab describes as "cognitive debt." This over-reliance threatens the future of learning and fosters a culture of academic dishonesty, with AI outputs often being undetectable after fine-tuning.

1. Societal Homogenization and Dilution of Labor Market: The pervasive use of GenAI leads to a broader problem of societal homogenization. When everyone uses AI for writing or creative tasks, the distinctiveness of human work diminishes. The inability to detect AI-generated content, especially after fine-tuning, erodes trustworthiness and allows AI to dilute the labor market. The speaker's own "current work" suggests that lay readers often prefer AI-written, watermarked outputs over human-written ones, highlighting a concerning trend where quality and authenticity are compromised.

1. Copyright Infringement and Rent-Seeking Behavior: Frontier AI companies engage in problematic practices regarding copyrighted material. Conflicting statements from companies like Meta (claiming copyrighted books have "no economic value") and Anthropic (shredding millions of physical books for training) reveal a fundamental contradiction. The lobbying efforts to train on copyrighted work without proper licensing or attribution are characterized as rent-seeking behavior. This not only harms creators but also raises questions about the ethical basis of large-scale AI model training, especially when AI outputs imitate creator styles, which current copyright law often fails to protect adequately.

1. Uneven AI Democratization and Data Colonization: The current AI safety paradigm is criticized for its restrictive view on long-term consequences, often overlooking the global inequities exacerbated by AI development. Uneven AI democratization leaves lower-income countries "data colonized" and dependent on external AI innovation, perpetuating a cycle of technological and economic subjugation rather than shared prosperity.

1. Productivity Gains vs. Actual Impact: While AI is often touted for productivity gains, real-world data suggests a more complex picture. The METEOR eval for developers, for instance, indicated that using AI actually made them spend more time, rather than increasing productivity. This challenges the widespread assumption that AI universally leads to efficiency, particularly for complex cognitive labor beyond basic software engineering or "lighter research."

These findings collectively argue that focusing solely on abstract, long-term catastrophic risks while neglecting these immediate and systemic harms is a misdirection for the AI safety community.

Technical Deep Dive

▶ Watch: Problem 2: Generative AI impairs learning and knowledge (3:30)

While this talk is a position paper rather than a presentation of novel technical research, it delves into the technical implications and design choices of AI systems that contribute to the identified problems. The discussion centers on the characteristics and deployment of foundation models, generative AI (GenAI), and AI agents, highlighting how their current architectural and operational paradigms exacerbate issues related to labor, learning, and ethics.

A critical technical point raised is the nature of foundation models themselves. These models are described as "closed form," meaning their internal workings, training data, and risk profiles are largely opaque to external scrutiny. Access is often limited to prompting through APIs, making it challenging for researchers, policymakers, and the public to truly understand and mitigate the risks they impose. This lack of transparency is a fundamental technical challenge, as it prevents comprehensive auditing and responsible development. The black-box nature of these powerful models contributes directly to the "irresponsible hype claims" surrounding their capabilities, as accurate technical statements without true access are difficult to verify.

The pervasive impact of generative AI on cognitive labor is a central theme. The talk implicitly touches upon the underlying architectures of these models, which are trained on vast datasets of text, code, and other modalities. The ability of GenAI to produce high-quality output for tasks like writing and coding is a direct consequence of these large-scale training regimes. However, this technical prowess leads to several problems:

• Cognitive Debt: The reliance on GenAI for problem-solving, particularly in educational settings, suggests that the models are effectively performing the cognitive work for the user. This bypasses the neural pathways and problem-solving strategies that would typically be developed through active learning, leading to a "cognitive debt" where individuals lose the ability to perform tasks without AI assistance. This points to a fundamental interaction design flaw in how GenAI is integrated into workflows, where it acts as a replacement rather than an augmentation tool.
• Undetectability of AI Output: A significant technical challenge highlighted is that fine-tuned GenAI outputs are often "not even detectable" as AI-generated, fooling both human readers and dedicated AI detectors. This implies that the stylistic and linguistic features learned during fine-tuning are sufficiently nuanced to mimic human authorship, making it difficult to distinguish authentic human creativity from machine replication. This has profound implications for trust, intellectual property, and the integrity of information.

The talk also addresses the technical practices surrounding training data. The controversy around copyrighted material being fed into AI models, exemplified by Anthropic's alleged shredding of physical books and Meta's stance on economic value, underscores a critical aspect of AI development. The effectiveness of large language models (LLMs) and GenAI models is directly proportional to the scale and diversity of their training data. Technically, these models learn patterns, styles, and information from this data to generate new content. The issue arises when this technical requirement for vast datasets clashes with existing legal and ethical frameworks for intellectual property. The lack of robust attribution mechanisms within the model architectures or output generation processes represents a technical gap that contributes to the problem of rent-seeking behavior and the dilution of the labor market. Designing alternative mechanisms for attribution and incentives for creators would require novel technical solutions embedded within the model training and inference pipelines.

Furthermore, the discussion on AI agents taking off "everywhere" implies a technical shift towards increasingly autonomous and integrated AI systems. These agents are designed to perform sequences of tasks, often automating complex cognitive labor. While the talk doesn't detail specific agent architectures, the implication is that their design prioritizes task completion and efficiency, potentially at the expense of human oversight, economic impact analysis, or ethical safeguards. The example of Grok-4 becoming "rogue" and generating "racist and sexist horrible things" points to inherent technical challenges in controlling and aligning the behavior of powerful, autonomous AI systems, especially when deployed at scale. This highlights the need for more robust alignment techniques and guardrail mechanisms in agent design.

In essence, while not presenting new algorithms, the technical deep dive in this position paper critiques the current technical trajectory of AI development, emphasizing how existing model design choices, training methodologies, and deployment strategies contribute to societal harms that warrant a re-prioritization within the AI safety framework.

Experimental Setup & Results

▶ Watch: Problem 4: Generative AI and copyright infringement (5:30)

As a position paper, the talk does not present novel experimental setups or original research results from the speakers' direct work, beyond a brief mention of "current work." Instead, it strategically leverages and synthesizes findings from various external studies, reports, and real-world observations to build a compelling evidence-based argument for its central thesis. These cited results serve as the empirical foundation for the claims regarding AI's impact on work, learning, and ethical considerations.

Key external evidence and findings cited include:

1. Anthropic's API Economic Index: This report indicates that approximately 50% of AI usage primarily concentrates on software development and writing tasks. This serves as a critical data point establishing the direct interface between AI capabilities and specific sectors of cognitive labor.

1. Harvard Business School Paper: A study from the Harvard Business School demonstrated that writing and coding jobs are facing a steep decline. When juxtaposed with Anthropic's index, this finding creates a direct correlation, suggesting that AI adoption in these areas is directly contributing to job displacement.

1. Student Performance with GPT-4: The talk references observations where students attempting to use GPT-4 as a "crutch" during practice math problems performed "a lot worse" when AI assistance was subsequently taken away. While not a formal experimental setup described in detail, this observation highlights a detrimental effect on genuine learning and skill acquisition, indicating that AI can hinder, rather than enhance, fundamental cognitive development in certain contexts.

1. Anthropic's Education Index/Report: This report further elucidates AI usage patterns in education, stating that more than 50% of AI output or usage in education is for "direct problem solving or direct output." This quantitative insight underscores the prevalence of AI being used as a shortcut rather than a tool for deeper engagement, reinforcing concerns about cognitive debt and cheating.

1. MIT Media Lab Paper: A recent paper from the MIT Media Lab is cited for its findings on "cognitive debt" resulting from the use of generative AI for learning and education. This conceptual result provides a theoretical framework for understanding the long-term negative impacts of AI over-reliance on mental faculties.

1. Speaker's "Current Work" on AI/Human Content Distinction: The speaker mentions their own ongoing research which "shows that if we train AI on copyrighted books then lay readers have a hard time even distinguishing what is AI written or what is human written." Furthermore, this work suggests that lay readers "actually policy prefer AI written watermarked outputs GenAI content over human written outputs which is not just scarier memorization but it definitely shows how the labor market is diluted." This constitutes an implicit experimental finding, indicating a measurable impact on perception and preference that directly affects the value and authenticity of human creative labor.

1. METEOR Eval for Developers: In the Q&A section, the speaker references the METEOR eval, which "shows that a lot of developers thought there's that this using AI actually makes them productive but it does not and it actually makes them spend more time." This directly challenges the widely held assumption of universal productivity gains from AI, particularly in complex cognitive tasks, suggesting that certain AI integrations can lead to decreased efficiency.

These cited results, drawn from diverse sources, collectively build a robust argument for the immediate and tangible impacts of AI on society. They serve to ground the position paper in empirical reality, contrasting with more speculative or theoretical discussions often found in AI safety. The absence of a bespoke experimental setup is characteristic of a position paper that aims to synthesize existing knowledge to advocate for a policy or research direction.

Practical Implications

▶ Watch: Policy recommendation: Watermark AI outputs, protect labor (6:30)

The position that AI safety should prioritize the future of work carries profound practical implications across various stakeholders, from policymakers and AI researchers to educators and individual workers. The talk delineates specific recommendations and highlights critical tradeoffs that must be navigated for responsible AI development and deployment.

For governments and policymakers, the implications are multi-faceted:

• Social Safety Nets and Worker Protections: There is an urgent need to expand and promote worker interests, establish comprehensive social safety nets, and implement policies that protect human labor from the disruptive forces of AI-driven automation. This includes exploring mechanisms for universal basic income or robust unemployment benefits tailored to an AI-impacted economy.
• Anti-Monopoly Measures: Governments must actively work to reduce the dominance of big tech monopolies in the AI space. This could involve antitrust actions, regulations on data access, or fostering competition to prevent a few large corporations from dictating the future of work and technological development.
• Copyright and Licensing Reform: Lawmakers must address the complex issue of copyrighted material in AI training. This necessitates considering collective licensing models, designing policies for attribution, and potentially mandating thorough ablation studies to assess the impact of copyrighted data on model performance. The goal is to create a framework that incentivizes creators while allowing for innovation.
• Responsible Regulation: While regulation is crucial, policymakers must navigate the delicate balance of preventing harm without stifling innovation. The talk acknowledges that "AI regulation could stifle innovation leading to other countries to gain a sort of dominance." This implies a need for nuanced, adaptive regulatory frameworks that are forward-looking and internationally coordinated.

For AI researchers and developers, the practical implications center on ethical responsibility and transparent practices:

• Consequence Awareness and Safeguards: Researchers must be acutely aware of the societal consequences of their work and actively deploy safeguards against irresponsible outcomes. This moves beyond abstract "alignment" to concrete measures that prevent job displacement, skill degradation, and the spread of misinformation.
• Accurate Capability Statements: A critical recommendation is to focus on accurate statements regarding AI capabilities, avoiding "irresponsible hype claims." This means rigorous evaluation, transparent reporting of limitations, and a commitment to scientific integrity over marketing narratives.
• Attribution and Ethical Training Data: Designing alternative positions for attribution and incentives for creators whose work forms the basis of AI models is paramount. This requires technical solutions embedded in the AI development lifecycle, ensuring fair compensation and recognition for human intellectual labor.
• Thorough Ablation Studies: Researchers should conduct thorough ablation studies to assess the impacts of copyrighted data on model behavior and output, providing empirical evidence for policy discussions.

For academic institutions and educators, the implications are about adapting learning environments:

• Updating AI Detection and Discourse: Institutions must update their beliefs on state-of-the-art AI detection, acknowledging the limitations of current tools. More importantly, they need to foster broader discourse on the longitudinal impact of GenAI and cognitive debt on students.
• Redesigning Educational Incentives: The traditional assignment cycle and assessment methods need a fundamental overhaul. Instead of relying on easily circumvented assignments, educators should explore approaches like "lectures and in-person exams" or other innovative pedagogical strategies that foster genuine learning and critical thinking, rather than enabling cheating and over-reliance on AI.

For practitioners, infra teams, model builders, and deployers, the key takeaway is to adopt a pro-worker governance framework. This means integrating ethical considerations, labor impact assessments, and equity principles into every stage of the AI lifecycle, from design to deployment. It requires a shift from purely optimizing for technical performance or profit to considering the broader societal impact, ensuring GenAI upholds shared prosperity and avoids becoming "a highway for labor displacement." The tradeoffs here involve balancing efficiency gains with social responsibility, potentially accepting slower development cycles or higher operational costs to ensure ethical outcomes.

Ultimately, the practical implication is a call for a holistic, systemic overhaul of how AI is conceived, developed, and governed, moving AI safety from a niche, theoretical concern to a central, actionable mandate for ensuring a just and prosperous future for human labor.

Key Takeaways

• Reframe AI Safety: The prevailing AI safety paradigm must expand its focus beyond hypothetical existential risks to prioritize the immediate, tangible impacts of AI on human labor, economic security, and the future of work.
• Combat Cognitive Debt: The widespread use of generative AI in learning and knowledge creation leads to "cognitive debt," impairing genuine skill development and fostering over-reliance that diminishes human capabilities.
• Address Market Dilution & Copyright: Unchecked AI training on copyrighted material without proper attribution or compensation perpetuates rent-seeking behavior, dilutes the labor market, and erodes trust in content authenticity, demanding urgent policy and technical solutions.
• Redesign Education: Academic institutions must fundamentally redesign educational incentives and assessment methods to counteract the negative effects of GenAI, fostering genuine learning rather than enabling shortcuts and intellectual dishonesty.
• Pro-Worker Governance: A pro-worker governance framework is essential for AI development, ensuring that technological advancements lead to shared prosperity, equitable distribution of benefits, and robust social safety nets, rather than exacerbating labor displacement and economic inequality.
• Transparency and Accountability: AI developers and researchers must prioritize transparency, accurate capability statements, and deploy safeguards to mitigate the societal risks of AI, especially in light of the "closed form" nature of many foundation models.

About the Speaker(s)

The talk was presented by Tuhin Chakrabarty, who delivered the position paper on behalf of himself and his co-authors, Sanchaita Hazra and Bodhisattwa Prasad Majumder. Tuhin Chakrabarty is actively involved in the academic community, particularly in research related to the societal impacts of AI. During the talk, he mentioned that he is currently recruiting students at Stony Brook CS for the upcoming fall, specifically seeking individuals interested in studying how generative AI impacts the labor market. This indicates his ongoing commitment to addressing the core themes of the presented position. While specific affiliations and titles for Sanchaita Hazra and Bodhisattwa Prasad Majumder were not detailed in the transcript, their co-authorship on this significant position paper underscores their shared expertise and concern regarding the ethical and practical implications of AI on human work and society.