LLM Privacy Paradox: Balancing Data Utility...
Rob Ragan, Aashiq Ramachandran
BSidesSF 2024 · Day 1
This article delves into the critical and often overlooked security implications of fine-tuning Large Language Models (LLMs), specifically focusing on the risk of **data leakage**. Presented by Rob Ragan and Aashiq Ramachandran at BSidesSF 2024, the talk highlights a fundamental paradox: while LLMs are expected to be trained on massive, diverse datasets, the presence of sensitive information within these datasets poses a significant risk. The core argument is that the underlying **Generative Pre-trained Transformer (GPT)** algorithm, being inherently statistical, lacks any built-in concept of security. Consequently, all efforts to make LLMs safe and secure must be diligently implemented by data scientists and security engineers throughout the model development lifecycle.
AI review
This talk provides a robust, experimental deep dive into the mechanics of data leakage in fine-tuned Large Language Models. The speakers systematically demonstrate how factors like data repetition and structural inconsistencies directly lead to memorization and sensitive information disclosure. They move beyond theoretical concerns to offer actionable insights into data preparation, model selection, and post-deployment mitigation strategies, including practical frameworks for sanitization and automated testing.