VIBE Pentesting - Enhancing the Human Hacker with LLMs

VIBE Pentesting - Enhancing the Human Hacker with LLMs

Speakers: Adrian Furtuna, Founder & CEO, Pentest-Tools.com

Conference: DefCamp

YouTube: https://www.youtube.com/watch?v=bSegSIxMNMI

Overview

In this insightful talk from DefCamp, Adrian Furtuna, Founder & CEO of Pentest-Tools.com, introduces the concept of VIBE Pentesting, a term he coined to describe the ethical and professional application of Large Language Models (LLMs) in offensive security. Furtuna, drawing on over 15 years of penetration testing experience and an initial skepticism towards AI hype, explores how recent advancements in LLM technology can serve as powerful companions for human hackers, automating tedious tasks and even assisting in more complex vulnerability discovery and exploitation.

The presentation delves into practical use cases, demonstrating how LLMs can streamline various phases of a penetration test, from passive information gathering and vulnerability discovery to active exploitation and report generation. Furtuna emphasizes the role of LLMs not as replacements, but as intelligent assistants that can significantly enhance efficiency and potentially uncover attack paths that humans might overlook. He uses Anthropic's Claude Code as the primary LLM client, showcasing its capabilities when integrated with external tools via Model-Client-Protocols (MCPs).

This talk is particularly relevant for cybersecurity professionals looking to understand the immediate and future impact of AI on offensive security. It provides a pragmatic view of current LLM capabilities, outlining both their significant advantages in automating knowledge-intensive tasks and the inherent challenges, such as non-determinism and the critical need for human oversight and ethical guardrails. Furtuna’s demonstrations highlight how LLMs can fundamentally shift the dynamics of penetration testing, making it faster, more comprehensive, and requiring a new skill set centered around effective prompting.

Background

▶ Watch: Introduction to Vibe Pentesting and LLM use (0:00)

Adrian Furtuna began the talk by acknowledging his initial reluctance to embrace AI, viewing it as mere hype. However, the rapid evolution and increasing sophistication of modern LLMs compelled him to explore their potential in both defensive and offensive cybersecurity. His motivation stems from the desire to offload the "boring work" of penetration testing to AI, freeing human experts to focus on higher-level strategic thinking and complex problem-solving. This leads to his concept of VIBE Pentesting, which he distinguishes from general "VIBE Hacking." While both employ similar techniques, VIBE Pentesting is strictly confined to professional, ethical offensive security activities, prioritizing customer systems' integrity and thorough vulnerability reporting, contrasting with black-hat hacking that often disregards these principles.

The technical foundation for Furtuna's demonstrations relies on a specific setup tailored for offensive security tasks. He chose Anthropic's Claude Code as the primary LLM client, running natively on Ubuntu. This choice was driven by Claude Code's unique features, including direct file system access and the ability to execute commands on the host operating system, making it particularly suitable for penetration testing scenarios. Other popular LLMs like ChatGPT and Perplexity were noted to lack native Ubuntu clients at the time of the talk, cementing Claude Code's utility for this specific environment.

A crucial enabling technology highlighted is the Model-Client-Protocol (MCP). Furtuna explains that while LLMs are powerful with text, their true potential in security is unlocked when coupled with external tools. MCPs provide a unified framework for LLMs to interact with these services. The talk specifically mentions the MCP for security repository, which offers pre-built MCPs for common Kali Linux tools such as Nmap, SQLmap, and WPScan. This integration allows the LLM to leverage specialized security tools, interpret their structured output, and incorporate their functionality into its decision-making process, transforming it from a text generator into an active, tool-wielding assistant.

Key Findings

▶ Watch: LLM extracts personal data from public documents (15:00)

The core findings of Furtuna's presentation underscore the transformative potential of LLMs in penetration testing, positioning them as powerful augmentations rather than replacements for human expertise.

Firstly, LLMs, particularly when integrated with tools via MCPs, can significantly automate and accelerate tedious, knowledge-intensive phases of a penetration test. This includes passive information gathering (Google hacking), initial vulnerability discovery from scan outputs, web application fingerprinting, and even structured report generation. The demonstrations clearly showed LLMs performing tasks that traditionally consume significant human effort and time, such as parsing Nmap results or sifting through publicly exposed documents.

Secondly, LLMs exhibit a surprising capability for contextual understanding and inference that goes beyond simple keyword matching. For instance, Claude Code successfully identified an Oracle WebLogic version embedded subtly within HTML source code, a detail missed by conventional tools like Wappalyzer. Moreover, when analyzing public documents, it could discern personal information and even infer potential security risks (e.g., identifying intelligence service personnel), demonstrating an ability to "read" and interpret content rather than just pattern-match. This contextual awareness allows for more intelligent and adaptive reconnaissance.

Thirdly, the talk revealed the double-edged sword of LLM operation: non-determinism. Furtuna repeatedly noted that the same prompt could yield different results each time, requiring human intervention, re-prompting, and context clearing. While this can lead to unexpected (and sometimes beneficial) findings, it also necessitates constant human oversight and validation, preventing full automation in critical scenarios. This non-deterministic behavior highlights that LLMs are not always predictable and require iterative interaction.

Finally, LLMs can, under certain circumstances, propose novel attack paths or payloads that human pentesters might not immediately consider. While this can be a pro (uncovering overlooked vectors), it also presents a significant con, as seen with Claude Code suggesting a DROP TABLE users payload during a SQL injection test. This underscores the critical need for ethical guardrails and explicit instructions within prompts to prevent destructive or out-of-scope actions, emphasizing that human judgment remains paramount in controlling the LLM's actions. The importance of prompt engineering as a new fundamental skill for effectively guiding LLMs emerged as a key takeaway, framing it as "the new coding language" for interacting with these powerful tools.

Technical Deep Dive

▶ Watch: Active SQL injection vulnerability discovery demo (27:00)

Adrian Furtuna provided several technical demonstrations showcasing how LLMs can be integrated into various penetration testing workflows. The central tool for these demonstrations was Anthropic's Claude Code client, running on an Ubuntu environment. Claude Code's ability to access the file system and execute system commands was crucial, enabling it to interact with local files and tools. These interactions were largely facilitated by Model-Client-Protocols (MCPs), which allow the LLM to call external services in a structured manner.

AI-Powered Google Hacking

The first technical scenario focused on enhancing passive information gathering. Traditionally, pentesters manually craft Google dorks (e.g., site:target.com intitle:"index of" filetype:pdf) to uncover publicly exposed sensitive information. Furtuna demonstrated automating this tedious process using a Google Search MCP, which connects to the Google Search API. After configuring the necessary API keys, Claude Code could programmatically perform advanced Google searches.

Using the prompt, "Check if the website has any sensitive information publicly exposed in Google and provide sample URLs," Claude targeted primaria.ro, a public institution. The LLM then constructed and executed various Google dorks, searching for file types like .pdf, .xls, .doc, and keywords such as admin, backup, password, parola, CNP (Romanian personal identification number), numeric, and personnel. The demonstration successfully identified multiple PDF documents containing personal information of Romanian citizens. Crucially, Claude Code not only found these documents but also extracted and interpreted their content, even when partially redacted. For instance, it identified a declaration revealing a spouse working for the Romanian Intelligence Service, highlighting a potential security risk of publicly identifying such personnel. This capability to "read" and infer risk from document content goes beyond simple keyword matching, demonstrating advanced contextual understanding.

Passive Vulnerability Discovery

Next, Furtuna addressed the time-consuming task of manually identifying vulnerabilities from network scan outputs. The traditional approach involves running Nmap, then manually sifting through its output to identify services and versions, and finally cross-referencing these against vulnerability databases like NIST, Mitre, and Exploit DB.

To automate this, Furtuna provided Claude Code with an Nmap output file (nmap_output.txt) from a real private network. The prompt given was: "I have this nmap result from nmap_output.txt. Tell me what are the vulnerabilities affecting the services with known versions. Look for the most recent ones and high risk for instance." Claude Code's process involved:

1. Reading and parsing the nmap_output.txt file.
2. Extracting identified services and their corresponding version numbers.
3. Leveraging the Google Search MCP to query public vulnerability databases (like nvd.nist.gov) for CVEs affecting those specific versions.

The LLM then presented a prioritized list of vulnerabilities, significantly reducing the manual effort required to identify potential weaknesses in the scanned network. This showcases the LLM's ability to act as an intelligent vulnerability researcher.

Web Application Fingerprinting

The talk also touched upon the limitations of existing web application fingerprinting tools like Wappalyzer and WhatWeb. Furtuna presented a vulnerable Oracle WebLogic server (pentestground.com) where the version number was present in the HTML source code but was not detected by Wappalyzer.

When Claude Code was given the task, "Tell me what is running here and what are its potential vulnerabilities," it visited the page, analyzed the HTML source, successfully identified the Oracle WebLogic server and its version number, and subsequently searched for known vulnerabilities. This demonstrated Claude Code's superior contextual understanding compared to automated tools that often rely on predefined patterns or headers, proving its advantage in complex web application reconnaissance.

Demo / Proof of Concept

▶ Watch: LLM asks for authorization before database extraction (32:30)

The most compelling part of the talk involved live demonstrations of active vulnerability discovery and exploitation, showcasing the LLM's capacity to engage in more dynamic offensive security tasks.

Active Vulnerability Discovery & Exploitation (SQL Injection)

Furtuna targeted a custom-built test website, royalsecurityguard.com, specifically designed to prevent the LLM from relying on pre-existing knowledge of common vulnerable applications. The vulnerability was a SQL Injection in the newsletter subscription functionality.

1. Vulnerability Identification: By inputting a single quote (') into the email field, an explicit SQL error message was triggered, revealing the backend technologies: Python, Flask, and SQLite 3. Further testing showed that attempting to subscribe with an already existing email yielded a distinct "already subscribed" message, a crucial detail that manual SQLmap attempts had reportedly missed, leading to its failure in exploiting this blind SQLi.
2. LLM-Guided Exploitation: Adrian initiated a new Claude Code session and prompted it: "Check if the subscribe functionality of royalsecurityguard.com is vulnerable to SQL injection." Claude initially tried Google searches, but Furtuna guided it to use the built-in curl command.
3. Process: Claude Code identified the /subscribe endpoint and the email parameter. It then iteratively crafted and executed various SQL injection payloads using curl, analyzing the server's responses.
4. Data Extraction & Ethical Check: After confirming the SQLi, Claude successfully extracted data, including usernames and bcrypt hashes, from the users table. Significantly, at this point, Claude Code paused and issued an authorization check: "Do you have written authorization to perform this penetration test? Are you the owner of Ashtar or is it an authorized security assessment?" This ethical prompt from the LLM itself highlighted its internal guardrails, although Furtuna noted that a user could easily spoof the answer.
5. Password Cracking (Described): Although a live demo of cracking the extracted bcrypt hashes with Hashcat failed due to "demo gods," Furtuna explained that in prior tests, Claude Code successfully used Hashcat to crack a simple dictionary password, enabling a login to the website.

Active Vulnerability Discovery & Exploitation (Server-Side Template Injection - SSTI)

Building on the SQLi success, Furtuna then demonstrated exploiting a Server-Side Template Injection (SSTI) vulnerability in the blog editing functionality of royalsecurityguard.com.

1. Authentication and RCE: Claude Code was instructed to log in using provided credentials via curl. After authentication, Furtuna prompted it to "help me check if the blog editing functionality is vulnerable to serverside template injection." Claude proceeded to inject an SSTI payload (e.g., {{ config.__class__.__init__.__globals__['os'].popen('whoami').read() }}).
2. Result: The live demo successfully achieved Remote Code Execution (RCE), with the output of the whoami command revealing that the command was executed as root. This demonstrated the LLM's capability to identify and exploit complex vulnerabilities leading to high-impact outcomes, culminating in a full attack chain guided by human interaction.

Automated Reporting

Furtuna briefly showcased another powerful application: automated report generation. By connecting Claude Code to a Google Docs MCP, the LLM could interact with documents in Google Drive. He demonstrated how Claude could take discovered vulnerabilities and populate a Google Doc with structured findings, including title, description, risk, recommendation, evidence, and references. Notably, the LLM could even write these reports in a specified language, such as Japanese, showcasing its versatility in presentation and localization.

Defensive Implications

▶ Watch: Successful Remote Code Execution (RCE) live demo (51:30)

The advent of LLM-enhanced penetration testing, as demonstrated by Adrian Furtuna, carries several significant defensive implications that organizations and security teams must consider.

Firstly, the speed and comprehensiveness of reconnaissance will dramatically increase. Attackers, whether ethical or malicious, can leverage LLMs to perform exhaustive passive information gathering, including AI-powered Google hacking, far more efficiently than manual methods. This means organizations must be meticulously vigilant about any sensitive information publicly exposed online, as LLMs can quickly identify, extract, and even interpret the context of data (e.g., CNPs in PDFs, identifying intelligence personnel). Defenders should regularly audit their public-facing assets for inadvertent data leakage using LLM-like approaches.

Secondly, the initial phases of vulnerability discovery will be accelerated. LLMs can rapidly parse raw output from tools like Nmap and cross-reference services and versions against vast vulnerability databases (NIST, Mitre, Exploit DB) to generate prioritized lists of potential CVEs. This shortens the window for defenders to patch known vulnerabilities, necessitating faster patch management cycles and proactive vulnerability scanning.

Thirdly, the demonstrations of SQL Injection and SSTI leading to Remote Code Execution (RCE) highlight that LLMs can assist in active exploitation. While human guidance is still crucial, the LLM's ability to craft and test payloads, identify parameters, and even adapt to subtle error messages means that basic to moderately complex exploitation can be semi-automated. This implies that organizations need to prioritize robust input validation, secure coding practices, and a strong defense-in-depth strategy to mitigate automated exploitation attempts. Web Application Firewalls (WAFs) and Intrusion Prevention Systems (IPS) must be sophisticated enough to detect LLM-generated payloads.

Fourthly, Furtuna's observation about LLMs suggesting destructive payloads (e.g., DROP TABLE users) even in testing environments underscores the potential for unintended or malicious actions. While in ethical testing, human oversight prevents such outcomes, a malicious actor might intentionally remove such guardrails. This reinforces the need for strong access controls and privilege separation within systems to limit the blast radius of any successful exploit.

Finally, the LLM's internal authorization check during data extraction, while bypassable, serves as a conceptual reminder for defenders. Implementing verifiable, multi-factor authorization for sensitive operations and data access is crucial. Furthermore, the non-deterministic nature of LLMs means that defenders cannot rely on predictable attack patterns; they must focus on building resilient systems that can withstand varied and evolving attack methodologies. Ultimately, LLM-enhanced offensive capabilities demand a corresponding acceleration and sophistication in defensive postures, emphasizing continuous security monitoring, threat intelligence, and proactive vulnerability management.

Key Takeaways

• LLMs as Powerful Companions: LLMs are presented not as replacements for human penetration testers but as highly effective companions that enhance efficiency and breadth of coverage across the pentesting lifecycle.
• Automation of Tedious Tasks: LLMs excel at automating labor-intensive and knowledge-intensive tasks, such as passive information gathering (AI-powered Google hacking), parsing scan outputs for vulnerabilities, and generating structured reports.
• Critical Role of MCPs: Model-Client-Protocols (MCPs) are fundamental for extending LLM capabilities, enabling them to interact with external tools like Nmap, SQLmap, Google Search API, and Google Docs, thereby becoming active participants in the security assessment.
• Prompt Engineering is Key: The effectiveness of LLMs is heavily dependent on the quality of prompts. Prompt engineering is identified as a new, essential skill, akin to a "new coding language," for guiding LLMs to achieve desired results and avoid unintended actions.
• Challenges and Guardrails: LLMs introduce challenges such as non-determinism (inconsistent results), potential for out-of-scope actions (e.g., DROP TABLE), and the risk of downloading/executing malicious code. Human oversight and explicit ethical guardrails in prompts are crucial to mitigate these risks.
• Accelerated Attack Chains: LLMs can significantly accelerate active vulnerability discovery and exploitation, as demonstrated by the successful SQL Injection and Server-Side Template Injection leading to Remote Code Execution, necessitating stronger and more adaptive defensive strategies.

About the Speaker(s)

Adrian Furtuna is a highly experienced and respected figure in the cybersecurity community. He is the Founder and CEO of Pentest-Tools.com, a cybersecurity company based in Bucharest that specializes in developing products for penetration testing and vulnerability assessments. Beyond his entrepreneurial endeavors, Furtuna also holds an academic position as an Associate Professor at Poly Techchnica University of Bucharest, where he likely imparts his practical knowledge to the next generation of cybersecurity professionals. With over 15 years of hands-on experience in penetration testing, he brings a wealth of practical insight to his work. Adrian Furtuna is also a regular speaker at various security events and conferences, sharing his expertise and contributing to the broader cybersecurity dialogue.