How to Talk AI to Your Lawyers

David Rosenthal (Lawyer)

Swiss Cyber Storm 2024 · Day 1 · Scenario

Overview

In his insightful talk at Swiss Cyber Storm, "How to Talk AI to Your Lawyers," David Rosenthal, a renowned legal expert with a deep understanding of artificial intelligence, addresses the critical communication gap between technology professionals and legal departments regarding AI deployment. Rosenthal argues that while AI presents unprecedented opportunities, its rapid adoption also introduces complex legal and ethical challenges that often overwhelm traditional legal frameworks and practitioners. The core of his presentation revolves around empowering technical teams to effectively engage with their legal counterparts, fostering a shared understanding of AI's technical underpinnings, and shifting the discourse from rigid compliance to pragmatic, risk-based management.

Watch on YouTube · Slides

Visual summary for How to Talk AI to Your Lawyers by David Rosenthal
Visual summary for How to Talk AI to Your Lawyers by David Rosenthal

Key moments

  1. 0:00 Welcome and purpose: Bridging AI and legal understanding
  2. 2:00 Understanding lawyers' fears and challenges with AI
  3. 6:00 Three strategies: Align, manage risk, invest in governance
  4. 10:00 AI Act's broad definition and practical irrelevance
  5. 14:00 Primary legal concerns: data protection, contracts, AI Act
  6. 20:00 Adopt a risk-based approach, not black-and-white compliance
  7. 25:00 Case study: The pitfalls of not consulting legal early
  8. 29:30 Switzerland's approach to AI regulation: fostering innovation

How to Talk AI to Your Lawyers

Speakers: David Rosenthal, Lawyer

Conference: Swiss Cyber Storm

YouTube: https://www.youtube.com/watch?v=0nF-ebEEM-Y

Overview

In his insightful talk at Swiss Cyber Storm, "How to Talk AI to Your Lawyers," David Rosenthal, a renowned legal expert with a deep understanding of artificial intelligence, addresses the critical communication gap between technology professionals and legal departments regarding AI deployment. Rosenthal argues that while AI presents unprecedented opportunities, its rapid adoption also introduces complex legal and ethical challenges that often overwhelm traditional legal frameworks and practitioners. The core of his presentation revolves around empowering technical teams to effectively engage with their legal counterparts, fostering a shared understanding of AI's technical underpinnings, and shifting the discourse from rigid compliance to pragmatic, risk-based management.

Rosenthal emphasizes that the legal landscape surrounding AI is fraught with uncertainty, fear, and a lack of specific, actionable guidance, often leading to either paralysis or risky unchecked deployments. He highlights that many lawyers, despite their best intentions, struggle to grasp the nuances of AI technologies like Large Language Models (LLMs) or neural networks, and are further burdened by the sheer volume of evolving regulations, such as the EU AI Act. This talk is crucial for any organization leveraging AI, as it provides a practical roadmap for navigating the legal complexities, mitigating potential liabilities, and accelerating responsible innovation by building bridges between technical capabilities and legal necessities.

The importance of this discussion cannot be overstated in an era where AI is no longer confined to specialized teams but is permeating every aspect of business operations. Rosenthal's unique perspective, bridging both legal and technical domains, offers an invaluable guide for security professionals, engineers, and business leaders seeking to integrate AI responsibly. By advocating for education, a risk-based approach, and robust governance, he aims to transform potential legal nightmares into manageable challenges, ultimately enabling organizations to harness AI's full potential without falling afoul of regulatory pitfalls or reputational damage.

Background

▶ Watch: Welcome and purpose: Bridging AI and legal understanding (0:00)

The rapid proliferation of Artificial Intelligence technologies, particularly generative AI systems like ChatGPT, has created a significant chasm between technological advancement and legal understanding. Historically, legal frameworks have struggled to keep pace with innovation, and AI presents an especially acute challenge due to its complexity, opacity, and rapidly evolving nature. Lawyers, as Rosenthal points out, are often overwhelmed by the volume of information in their own legal fields, let alone the intricacies of AI. This creates a situation where legal departments may react with fear, excessive caution, or even outright resistance to AI initiatives, driven by headlines proclaiming "uncontrolled AI legal nightmares."

Prior to the widespread adoption of AI, legal discussions around technology often centered on well-defined areas like data protection (GDPR being a prime example) or intellectual property. However, AI introduces new dimensions, such as algorithmic bias, explainability, memorization, and the potential for autonomous decision-making, which do not fit neatly into existing legal categories. The EU AI Act, while an ambitious attempt to regulate AI, is itself a complex document that many struggle to fully comprehend, even its drafters. Rosenthal notes that its definition of AI is so broad that it could encompass a 20-year-old copy machine with Optical Character Recognition (OCR), highlighting the disconnect between legislative intent and technical reality.

The problem is exacerbated by a lack of common vocabulary and understanding. Engineers often speak in terms of models, probabilities, and architectures, while lawyers focus on compliance, liability, and precedent. This communication barrier leads to missed opportunities, stalled projects, and, in some cases, significant legal exposure when AI applications are deployed without adequate legal foresight. The speaker illustrates this by mentioning that lawyers often don't trust their own knowledge of AI, which contributes to their conservative stance. The historical precedent of legal struggles with cloud computing adoption, though less pervasive across an organization, offers a parallel where legal and technical teams had to bridge gaps regarding data residency, security, and contractual terms. However, AI's impact is far more widespread, affecting nearly every department and increasing the pressure on organizations to manage these risks effectively.

Key Findings

▶ Watch: Three strategies: Align, manage risk, invest in governance (6:00)

David Rosenthal's talk delivers several key findings and actionable strategies for bridging the gap between AI development and legal compliance:

  1. The Emotional and Knowledge Gap in Legal: Lawyers often react to AI with fear and uncertainty, stemming from a lack of deep technical understanding and an overwhelming amount of information. They frequently don't trust their own knowledge of AI. This emotional response, combined with the perception that AI creates business (e.g., litigation), often leads to a default position of caution or obstruction.
  2. Shift from Compliance to Risk Management: The traditional "is it compliant or not?" black-and-white approach is ineffective for AI. Instead, organizations must adopt a risk-based approach, focusing on identifying, assessing, and managing the residual risks associated with AI deployments. Lawyers are often more open to this nuanced approach than technical teams might assume, provided the business is willing to take calculated risks.
  3. The Imperative of Mutual Education: Both technical and legal teams need to educate each other. Engineers must explain the technical concepts of AI (e.g., how neural networks and LLMs work, memorization issues, the difference between generative AI and analytical AI) to lawyers. Conversely, technical teams must understand the specific legal triggers and concerns of lawyers, such as those related to personal data protection (GDPR), confidentiality, and automated decisions.
  4. Misconceptions about AI Act and Copyright: Rosenthal clarifies that the EU AI Act is primarily a product safety regulation, not a general AI regulation, and will likely only apply to 5-10% of high-risk systems. Similarly, copyright issues, while often cited by lawyers, are not a major practical concern for most companies outside the content industry. These insights help focus legal efforts on more relevant areas.
  5. The Challenge of Ambiguous Legal Terms: Concepts like "explainability" in AI are often demanded by lawyers without a clear understanding of what they practically entail or how they can be achieved in complex AI systems. Rosenthal argues that many legal terms in AI regulations lack established standards and will take years to develop concrete meanings, necessitating a pragmatic, context-specific approach rather than rigid adherence to ill-defined concepts.
  6. The Necessity of AI Governance: Investing in a robust AI governance framework is critical. This includes defining what constitutes "AI" within the organization, establishing risk categories, and implementing processes for vetting AI applications. While potentially bureaucratic, governance provides a structured approach to managing legal and reputational risks, ultimately empowering responsible AI adoption.

Technical Deep Dive

▶ Watch: Primary legal concerns: data protection, contracts, AI Act (14:00)

While the talk is primarily focused on legal communication, Rosenthal delves into several technical concepts that lawyers need to understand to properly assess AI risks. He argues that it's not enough for lawyers to know that AI exists; they must grasp how it fundamentally operates.

He begins by explaining the basic mechanics of Large Language Models (LLMs) and neural networks, emphasizing that they "estimate probabilities." For lawyers, understanding that LLMs don't "store" information in the traditional sense, but rather learn patterns and relationships, is crucial for dispelling fears about memorization. While he acknowledges that LLMs can memorize specific data points, he clarifies that "it's not that everything is stored in there and is getting out." This distinction helps lawyers understand the actual risk profile regarding data leakage or privacy breaches, rather than assuming a direct database-like storage mechanism.

Rosenthal challenges the broad, often ill-defined scope of "AI" in legal definitions, particularly citing the OECD and EU AI Act definitions. He points out that under these definitions, a simple copy machine with Optical Character Recognition (OCR), a technology that has existed for decades and uses basic neural networks for character recognition, would be classified as AI. This highlights the practical absurdity of applying broad AI regulations to mature, low-risk technologies. He stresses that organizations need to internally define what "AI" means for their governance programs, distinguishing between systems like random forests (clearly AI) and linear regression (more ambiguous but still potentially impactful in regulated sectors like insurance).

He also touches upon the security implications of AI, urging technical teams to educate lawyers on risks beyond just inaccurate outputs. For instance, prompt injection attacks against chatbots that could lead to unauthorized access to sensitive customer data stored in a Retrieval-Augmented Generation (RAG) database are far more critical security concerns than a chatbot simply providing a wrong response. This illustrates that the "technical deep dive" for lawyers isn't about writing code, but about understanding the attack vectors and underlying mechanisms that create legal and security risks.

Furthermore, Rosenthal addresses the concept of explainability, a frequent demand from legal teams. He notes that lawyers often request "precise explanations" for LLM outputs without understanding the inherent difficulty, if not impossibility, of providing a human-interpretable step-by-step breakdown for complex neural networks. He suggests alternative, more pragmatic approaches to explainability, such as running two different models against each other and seeing if they arrive at similar, human-sensible conclusions. He likens this to human perception: "we humans also cannot explain if you see a cat somewhere you immediately it's a cat but you can't explain why you can explain afterwards why it makes sense that you say this is a cat." This analogy helps demystify the technical limitations and encourages a more realistic expectation from legal teams.

Finally, the talk implicitly covers the technical challenges of data protection in AI models. Rosenthal mentions creating charts for lawyers to illustrate "how does personal data end up in a model or not, what is really stored in there, how do these concepts work." This involves understanding data pipelines, training data curation, anonymization techniques, and the technical limitations of removing specific data points from trained models, all of which are critical for addressing GDPR and other privacy regulations.

Demo / Proof of Concept

▶ Watch: Adopt a risk-based approach, not black-and-white compliance (20:00)

While David Rosenthal's talk does not feature a live technical demonstration or a traditional security Proof of Concept (PoC), he effectively uses a real-world case study to illustrate the severe consequences of failing to engage legal teams early and effectively in AI projects. This serves as a "proof of concept" for the problem his talk addresses – the legal and reputational fallout from unvetted AI deployments.

The speaker recounts a widely publicized incident involving a large Swiss insurance company that allegedly deployed a device in its offices. This device was designed to "listen in" and determine "how often do the employees laugh." If the laughter frequency was deemed insufficient, an AI system would then trigger the sending of jokes via email to employees, with the stated goal of making them "more happy."

Rosenthal highlights this as a "legal nightmare" for several reasons:

  1. Prohibited Activity under AI Act: He points out that this scenario squarely falls under one of the few explicitly prohibited activities in the EU AI Act: "emotion recognition at the workplace." This demonstrates a clear violation of emerging AI regulations.
  2. Lack of Legal Involvement: Crucially, the legal department of the insurance company was completely unaware of this initiative. Rosenthal personally spoke with one of the company's lawyers, who was "absolutely mad" upon learning about it during his vacation, underscoring the lack of internal communication and governance.
  3. Reputational Damage: Even if the system was not fully deployed (as the lawyer suggested), the public perception and media headlines created significant reputational damage. This illustrates the "storm" that can be generated when AI projects are perceived as unethical or invasive, regardless of their actual operational status.

This anecdote serves as a potent example of what happens when the principles outlined in the talk—early legal engagement, understanding legal triggers (Biometrics, emotion recognition, employees), and implementing proper governance—are neglected. It underscores the financial, legal, and reputational risks that organizations face when technical teams proceed with innovative AI ideas without the necessary legal vetting and risk assessment. The "demo" here is a cautionary tale, vividly illustrating the real-world implications of the communication breakdown that the talk aims to prevent.

Defensive Implications

▶ Watch: Switzerland's approach to AI regulation: fostering innovation (29:30)

The defensive implications derived from David Rosenthal's talk are not about patching vulnerabilities in code, but about establishing robust organizational defenses against legal, regulatory, and reputational risks associated with AI. For security professionals and organizations deploying AI, the advice translates into a strategic shift in how they interact with legal departments and manage AI projects.

  1. Prioritize Mutual Education: Security teams must proactively educate their legal counterparts on the technical realities of AI. This includes explaining concepts like neural networks, LLM memorization, prompt injection attacks, and the difference between generative AI and analytical AI. Conversely, security professionals should understand the legal team's primary concerns, such as data protection (GDPR), confidentiality, automated decision-making, third-party service provider contracts (e.g., Microsoft Azure, Google, OpenAI), and specific regulatory triggers like biometrics or emotion recognition at the workplace.
  2. Adopt a Risk-Based AI Strategy: Move away from a black-and-white "compliant or not" mindset to a risk-based approach. Define clear risk categories for different AI applications within the organization. This allows for focusing resources on high-risk systems (e.g., 5-10% under the EU AI Act) while enabling faster deployment for lower-risk applications. Security teams should develop a risk matrix to help legal departments understand and accept calculated risks, empowering the "first line of defense" (business/tech) to make informed decisions.
  3. Invest in Comprehensive AI Governance: Implement a formal AI governance framework. This includes:
  • Defining AI Scope: Clearly define what constitutes "AI" within the company to determine which systems fall under governance.
  • Policy Development: Establish internal policies and guidelines for AI development, deployment, and usage, covering aspects like data handling, model validation, and output monitoring.
  • Tool Vetting: Vet external AI tools and providers (e.g., ChatGPT, Azure, Google, OpenAI) to ensure their contracts and data handling practices align with internal standards and legal requirements, offering employees approved "sandboxes" for experimentation.
  • Early Legal Engagement: Mandate early involvement of legal counsel in all AI projects, especially those touching sensitive areas like personal data, employees, or automated decisions. This prevents costly retrofitting or public relations disasters, as exemplified by the Swiss insurance company case.
  1. Focus on Practical Legal Concerns: While lawyers may raise concerns about copyright or broad interpretations of the AI Act, security teams should guide the conversation towards more immediate and impactful legal risks. Data protection remains the paramount legal issue for most companies using AI. Focus on demonstrating robust controls for data retention, anonymization, and consent. Also, emphasize the security aspects of AI, such as preventing prompt injection that could expose customer data, as these often present greater practical risks than perceived issues like hallucination.
  2. Manage Expectations for Explainability: Acknowledge that perfect explainability for complex AI models like LLMs is often unachievable. Instead, focus on demonstrating transparency through alternative means, such as model validation, performance monitoring, and comparative analysis (e.g., running two models against each other). Help legal teams understand what practical "comfort" can be provided regarding AI decision-making without demanding impossible technical feats.
  3. Understand and Mitigate Legal Triggers: Be acutely aware of "red flag" legal triggers that immediately raise concerns for lawyers. These include:
  • Biometric data
  • Emotion recognition
  • AI systems dealing with employees (e.g., performance monitoring, hiring)
  • Automated decisions with significant impact on individuals
  • Reliance on third-party service providers without proper contractual vetting.

By proactively addressing these areas or designing systems to avoid them where possible, organizations can significantly reduce legal friction.

Key Takeaways

  • Bridge the Communication Gap: Actively educate legal teams on AI's technical fundamentals (e.g., LLMs, neural networks, memorization) and security aspects (e.g., prompt injection), while understanding their legal concerns (data protection, automated decisions).
  • Embrace Risk-Based AI Management: Shift from absolute compliance to a pragmatic, risk-based approach for AI deployment, defining risk categories and accepting manageable risks, rather than seeking impossible legal certainty.
  • Prioritize Robust AI Governance: Implement a comprehensive AI governance framework, including clear internal definitions of AI, established policies, and vetting processes for AI tools and providers, to ensure responsible and compliant innovation.
  • Focus on Real, Not Perceived, Legal Risks: Recognize that data protection is the primary legal concern for most AI applications, while issues like copyright and the broad applicability of the EU AI Act are often overemphasized for general use cases.
  • Engage Legal Early and Often: Involve legal departments at the earliest stages of AI project conceptualization, especially for systems touching sensitive areas like biometrics, emotion recognition, or employee data, to prevent costly legal and reputational fallout.
  • Manage Explainability Expectations: Understand that perfect "explainability" for complex AI is often unrealistic; instead, focus on practical transparency measures and help legal teams understand what comfort can realistically be provided.

About the Speaker(s)

David Rosenthal is a distinguished lawyer widely recognized for his expertise at the intersection of law and artificial intelligence. He is lauded as a leading expert in the legal aspects of AI, possessing a unique understanding that bridges both the legal and technical domains. Unlike many legal professionals, Rosenthal makes a conscious effort to delve into the underlying mathematics and concepts of AI, such as how Large Language Models (LLMs) and neural networks function. This deep technical insight allows him to provide nuanced and practical legal advice that resonates with both engineers and legal practitioners. He is also known for actively sharing his knowledge and frameworks on platforms like LinkedIn, where his posts garner significant engagement, reflecting the value and relevance of his contributions to the field. His background enables him to articulate the challenges lawyers face in comprehending AI while simultaneously guiding technical professionals on how to effectively communicate and collaborate with their legal counterparts.

Reviews

Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT

I usually avoid legal policy talks like the plague, but Rosenthal actually understands what a neural net is. He strips away the copyright FUD and EU AI Act hysteria to focus on how you actually unblock engineering teams when general counsel is panicking. If you are tired of lawyers treating your RAG pipeline like a GDPR liability, watch this.

Heather Calloway (CISO) — STRONG ACCEPT

Every CISO is currently fighting a two-front war between a business that wants to deploy LLMs yesterday and a general counsel who wants to ban them until the EU AI Act makes sense. Rosenthal provides a practical framework for breaking that deadlock. Required viewing for anyone trying to get an AI governance committee off the ground without stalling the business.

→ Top-rated talks at Swiss Cyber Storm 2024

All talks from Swiss Cyber Storm 2024