Cracks-in-Agent-Collaboration

This is the official repository for paper Cracks in Collaboration: Threat Models and Attacks on Multi-LLM Collaborative Systems published in IEEE Transactions on Dependable and Secure Computing.

The paper investigates the emerging security risks in collaborative multi-agent LLM systems, where multiple language models interact, coordinate, and exchange information to solve complex tasks.
We systematically analyze the threat surface introduced by internal communication and propose novel attack strategies that exploit collaboration mechanisms, including malicious prompt injection (tool-call), information stealing, response manipulate. The work further presents a comprehensive threat model, empirical evaluations across three representative collaboration frameworks (centralized framework, horizontal framework and joint framework), and insights into the security implications of autonomous LLM collaboration.

This repository contains the implementation and resources for reproducing the results presented in the paper.

citation

If you find our work is helpful, please leave us a star

@ARTICLE{Yang2026Cracks,
  author={Yang, Meng and Zhu, Tianqing and Liu, Bo and Xu, Heng and Zhou, Wanlei},
  journal={IEEE Transactions on Dependable and Secure Computing}, 
  title={Cracks in Collaboration: Threat Models and Attacks on Multi-LLM Collaborative Systems}, 
  year={2026},
  volume={23},
  number={3},
  pages={7191-7207},
  doi={10.1109/TDSC.2026.3670889}}

Multi-LLM Collaborative framework

Centralized

There will be a server LLM and several client LLMs in this framework.

Server distribute the task to clients and clients return their response to the server. The server solve the task based on the client responses.

Horizontal

Each LLM are linked one-by-one, where the first LLM get the user query and generate a response.

Then the query and response are sent to the next LLM until the final one.

Joint

Each LLM can freely communicate with other LLMs.

Communication Process

Direct: Each message will directly concatenate as the new message as $K^{i+1} = K^i_{cc_0} \oplus K^i_{cc_1}, \oplus \dots \oplus K^i_{cc_n}.$

Summary: model summary the previous message as the new message as $K^{i+1} = summary(K^i_{cc_0}, K^i_{cc_1}, \dots, K^i_{cc_n}).$

Vote: model selects one piece of provided message as the most suitable and uses it as the updated message for the next round as $\exists K^{i+1} \in (K^i_{cc_0}, K^i_{cc_1}, \dots, K^i_{cc_n}).$

Attack Framework

Decision Poisoning Attack (AS-I)

Indirect EchoLeak Attack (AS-II)

Information Collision Attack (AS-III)

Step1

Create virtual environment

conda env create -f environment.yml

Step2

run the code

bash run.sh

This code will run each script in folder ./centralized ./horizontal and ./joint

Case Study

Contact

Any question please contact yangmengjlu@gmail.com.