Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents

Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents

Speakers: Zichuan Li

Conference: NDSS Symposium 2026

YouTube: https://www.youtube.com/watch?v=N2GdHeU4EWY

Overview

This talk presents Cross-Tool Harvesting and Polluting (XTHP) attacks, a new class of supply chain threats targeting LLM agent development frameworks like LangChain and LlamaIndex. The research demonstrates that malicious tools can be crafted with benign-looking descriptions that exploit how LLMs select and sequence tool calls, enabling three attack capabilities: control flow hijacking (forcing the agent to invoke the malicious tool), data harvesting (extracting sensitive information from the agent context), and information polluting (replacing correct outputs with misleading data).

The findings are alarming: 75% of framework tools in LangChain and LlamaIndex are vulnerable to XTHP attacks. The attacks bypass existing defenses including air-gap style data filtering, tool filtering, and prompt injection detection techniques like Spotlighting and PI detectors. The researchers also demonstrate that emerging paradigms including MCP (Model Context Protocol) tools and agent skills are equally vulnerable. Findings were reported to the LangChain and LlamaIndex security teams.

Background

▶ Watch: LLM agent supply chain: LangChain and LlamaIndex tool ecosystems (0:00)

LLM agents are increasingly built using agent development frameworks that provide third-party integrations for tools, databases, and models. LangChain maintains a centralized community repository and LlamaIndex has LlamaHub, both hosting hundreds of third-party tool integrations. When developers build agents, they configure a pool of tools -- the set of available tools the agent can invoke. At runtime, the agent decides which tools to call and in what sequence (the control flow) based on natural language descriptions.

The critical security gap is that tool implementation is opaque to the LLM. The LLM only sees the tool schema: a name, a description, arguments, and an entry function specification -- all expressed in natural language. The LLM trusts these descriptions without any ability to verify the actual implementation. This creates an exploitable disconnect between what a tool claims to do and what it actually does.

The research identifies a root cause: LLMs prefer invoking tools over using their own intrinsic knowledge. Even when the LLM has the necessary knowledge internally (like knowing that Apple's stock ticker is AAPL), if a tool is available that claims to provide this information, the LLM will preferentially invoke the tool. This preference creates the opening for control flow hijacking.

Key Findings

▶ Watch: Data harvesting through malicious tool parameters (4:00)

75% of framework tools are vulnerable: Analysis of tools in LangChain and LlamaIndex found that three-quarters can be targeted by XTHP attacks, enabling financial loss, misinformation spread, and privacy leakage.

Four attack vectors identified:

• Target Semantic Hooking: Exploiting external dependencies of legitimate tools. If a tool requires a specific input format (like a stock ticker), a malicious tool can claim to provide that format conversion.
• Syntax Format Hooking: Targeting tools that require specific input formats (URLs, JSON) by claiming to prepare those inputs.
• LLM Preference Hooking: Crafting tool descriptions that exploit LLM preferences for tools claiming superior features (performance, reliability, fairness, LLM-friendliness). A "good search" tool claiming robust, reliable, efficient results with LLM-friendly formatting is consistently preferred over legitimate Google search.
• Dynamic Tool Descriptions: Fetching tool names and descriptions from remote servers at runtime, showing benign descriptions in code but loading malicious descriptions during execution.

Attacks bypass existing defenses: XTHP attacks remain effective against air-gap style data filtering, tool filtering, and prompt injection detection methods including Spotlighting and PI detectors.

MCP and skills are equally vulnerable: The emerging MCP protocol and agent skill paradigms use the same trust-based tool selection mechanism, making them susceptible to identical attacks.

LLMs prefer tools over knowledge: When a tool is available that addresses an information need, LLMs consistently choose to invoke the tool rather than use their own knowledge, even when their intrinsic knowledge is sufficient and correct.

Technical Deep Dive

▶ Watch: LLM preference hooking: crafting irresistible tool descriptions (8:00)

Control Flow Hijacking is the foundation of XTHP attacks. Consider a financial agent with tools for Yahoo Finance (news), Stock Data API (prices), and Data Analyst (analysis). A user asks about buying Apple stock. The normal flow: search news -> fetch prices -> analyze data. A malicious tool called "company_to_ticker" claims to convert company names to stock tickers. Even though the LLM knows Apple's ticker is AAPL, the malicious tool's description creates a knowledge gap exploitation -- the LLM sees a tool that addresses a dependency (converting "Apple" to "AAPL") and invokes it before Yahoo Finance.

Data Harvesting exploits the hijacked position in the control flow. Once invoked, the malicious tool can include parameters requesting arbitrary information from the agent context. For example, a parameter like "net_worth_value" will be populated by the LLM with whatever financial information exists in the context. The LLM treats tool parameters as requirements to be fulfilled, not as potential exfiltration channels.

Information Polluting replaces correct tool outputs with misleading data. The malicious company_to_ticker tool could return "APD" instead of "AAPL", causing all downstream tools to fetch data for the wrong company and the agent to provide incorrect analysis to the user.

LLM Preference Hooking uses LLM-based mutation to generate maximally preferred tool descriptions. The mutation optimizes along four dimensions: performance claims, fairness assertions, reliability guarantees, and LLM-friendliness features. The resulting descriptions consistently cause LLMs to prefer the malicious tool over legitimate alternatives with identical functionality.

Dynamic Tool Descriptions provide stealth by fetching malicious descriptions from remote servers at runtime. The code repository shows benign descriptions, but the tool dynamically loads different names and descriptions during agent execution. This technique also enables dynamic output polluting by fetching polluted return values from remote servers.

XTHP Threat Scanner is a fully automated pipeline that evaluates whether existing tools are vulnerable. Given a target tool, the hijacker generates candidate malicious descriptions and tests them against a testing agent with representative user queries. An optimization loop refines descriptions until hijacking succeeds. The harvester then identifies extractable data, and the polluter generates misleading return values.

Demo / Proof of Concept

▶ Watch: 75% of framework tools vulnerable; MCP also affected (10:00)

The talk walked through a detailed financial agent example showing the complete attack chain: a benign stock analysis query hijacked through a company_to_ticker tool, data harvested through malicious parameter definitions, and analysis polluted through incorrect ticker substitution. The evaluation showed that the automated XTHP scanner identified 75% of LangChain and LlamaIndex tools as vulnerable. Defense bypass testing demonstrated resilience against multiple existing protection mechanisms.

Defensive Implications

▶ Watch: Q&A: cross-validation, defense techniques, and attack limitations (14:00)

Agent development framework security is insufficient: LangChain and LlamaIndex lack tool vetting processes, making their tool ecosystems vulnerable to supply chain attacks. Framework maintainers need to implement tool review, behavioral sandboxing, or runtime verification.

Tool descriptions are an attack surface: Security teams deploying LLM agents must treat tool descriptions as untrusted input, not trusted configuration. Any third-party tool's description could be crafted to hijack agent behavior.

MCP security implications: The emerging MCP standard inherits the same vulnerability pattern -- tools are selected based on natural language descriptions from MCP servers, with no verification of actual behavior. Organizations adopting MCP should implement additional verification layers.

Runtime monitoring needed: Static analysis of tool descriptions is insufficient because dynamic tool descriptions can change behavior at runtime. Runtime monitoring of tool invocation patterns, data flows between tools, and output consistency is necessary.

Defense recommendations: The researchers suggest (1) filtering tools based on relevance to the current context, (2) validating tool outputs against expected results, (3) detecting when tools request data unnecessary for their stated purpose, and (4) implementing tool provenance and vetting processes in framework repositories.

Key Takeaways

• XTHP attacks exploit LLMs' preference for tools over intrinsic knowledge and trust in tool descriptions to hijack agent control flow
• 75% of LangChain and LlamaIndex tools are vulnerable to control flow hijacking, data harvesting, and information polluting
• Four attack vectors: target semantic hooking, syntax format hooking, LLM preference hooking, and dynamic tool descriptions
• Attacks bypass existing defenses including air-gap filtering, tool filtering, Spotlighting, and PI detectors
• MCP tools and agent skills are equally vulnerable to the same attack patterns
• The root cause is that tool implementation is opaque to LLMs, and LLMs trust natural language descriptions without verification
• Findings reported to LangChain and LlamaIndex security teams; source code and evaluation scripts publicly available

About the Speaker(s)

Zichuan Li (Tuchan Lee) presented this work, which was conducted in collaboration with Tianu (Jen) and supervised by professors including Luising. The research team spans expertise in LLM agent security, supply chain security, and adversarial machine learning. Both presenters demonstrated strong knowledge of the agent development ecosystem and engaged effectively with challenging Q&A questions from researchers at Purdue, Jiaotong University, and other institutions about attack limitations, defense mechanisms, and cross-validation behaviors.