One-Click Code Fix: Securing Code Using AI
Chandrani Mukherjee, Joseph Seasly
BSidesSF 2024 · Day 1
This talk, presented by Chandrani Mukherjee and Joseph Seasly at BSidesSF 2024, explores the ambitious goal of leveraging Artificial Intelligence (AI) to automatically identify and remediate code vulnerabilities. The core premise is that AI can process code at speeds unmatched by humans, offering a potential paradigm shift in how security teams manage code vulnerabilities. However, the speakers immediately temper expectations, noting that AI-generated fixes are far from perfect, citing a benchmark where even **ChatGPT-4** achieved only a 2% success rate for complex bug fixes. This underscores the critical need for human oversight and strategic integration of AI into existing security workflows.
AI review
This talk provides a pragmatic and technically grounded journey into leveraging LLMs for automated code vulnerability fixing. The speakers detail their iterative process, from initial zero-shot attempts to the more effective Chain of Thought prompting, highlighting both successes and significant challenges. It's a solid engineering effort that offers actionable insights for teams looking to integrate AI into their secure development lifecycle, emphasizing the critical role of human oversight and robust evaluation frameworks.