Exploit Tool Invocation Prompt for Tool Behavior Hijacking in LLM-based Agentic System

Yu Liu, Yuchong Xie, Mingyu Luo, Zesen Liu, Zhixiang Zhang, Kaikai Zhang, Zongjie Li, Ping Chen, Shuai Wang, Dongdong She

Institutions: Fudan University, The Hong Kong University of Science and Technology

Equal contribution | Corresponding author

Introduction

LLM-based agentic systems leverage large language models to execute external tools for complex, multi-step tasks. These systems are widely used in domains such as chatbots, customer service, and software engineering. A key component of these systems is the Tool Invocation Prompt (TIP), which defines tool-interaction protocols and security behaviors.

Despite its importance, the security of TIPs has been overlooked. This paper investigates the vulnerabilities of TIPs, revealing critical risks such as Remote Code Execution (RCE) and Denial of Service (DoS).

Methods

We propose a systematic TIP Exploitation Workflow (TEW) consisting of:

Step 1: Prompt Stealing

Extract system prompts using adversarial instructions embedded in tool descriptions.

Step 2: TIP Vulnerability Analysis

Identify vulnerabilities in tool descriptions, formats, and execution pathways.

Step 3: TIP Hijacking

Exploit vulnerabilities to manipulate tool invocation protocols and achieve RCE or DoS.

Methods Overview

Figure 1: Methods Overview

Results

Our empirical evaluation across multiple LLM-based systems demonstrates:

Vulnerable systems include Cursor, Claude Code, and other IDE and CLI agents.

Case Study 1: RCE on Cursor

Figure 2: Case Study 1 - RCE on Cursor

Case Study 2: RCE on Claude Code

Figure 3: Case Study 2 - RCE on Claude Code

Case Study 3: DoS on Cline

Figure 4: Case Study 3 - DoS on Cline

Conclusion

TIPs are critical yet vulnerable components of LLM-based agentic systems. Our findings highlight the need for robust security measures, such as layered defenses and adaptive filtering, to mitigate risks of RCE and DoS.

@misc{liu2025exploittoolinvocationprompt, title={Exploit Tool Invocation Prompt for Tool Behavior Hijacking in LLM-Based Agentic System}, author={Yu Liu and Yuchong Xie and Mingyu Luo and Zesen Liu and Zhixiang Zhang and Kaikai Zhang and Zongjie Li and Ping Chen and Shuai Wang and Dongdong She}, year={2025}, eprint={2509.05755}, archivePrefix={arXiv}, primaryClass={cs.CR}, url={https://arxiv.org/abs/2509.05755}, }