# Poisoned LangChain: Jailbreak LLMs by LangChain

**Authors:** Ziqiu Wang, Jun Liu, Shengkai Zhang, Yang Yang  
**arXiv:** [2406.18122](https://arxiv.org/abs/2406.18122)  
**Date:** 2024-06-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.18122v1/

## Key points

- Large language models (LLMs) have become widely integrated into everyday life, raising concerns about security vulnerabilities.
- Safeguarding the security of LLMs is a critical priority due to evolving threats in attacking and defending against them.
- Jailbreak attacks are a significant threat aimed at circumventing model safety mechanisms to generate inappropriate or malicious content.
- Traditional jailbreak tactics may be less effective against robustly filtered and highly comprehending large models.
- Retrieval-Augmented Generation (RAG) addresses the need for continuous updates by enabling LLMs to leverage external knowledge bases but also introduces new risks for potential jailbreak attacks.
- The concept of indirect jailbreak is introduced through Poisoned-LangChain (PLC), which leverages poisoned external knowledge bases to prompt large language models to generate non-compliant or malicious dialogues inadvertently.
- Experiments conducted by researchers demonstrated that PLC successfully executed indirect jailbreak attacks with success rates ranging from 79.04% to 88.56% across different large language models and categories of jailbreak issues.
- Developing robust defense mechanisms is crucial to safeguard against sophisticated attacks faced by large language models in an interconnected digital world.

## Summary

In the rapidly evolving landscape of natural language processing (NLP), large language models (LLMs) have gained significant traction, permeating various aspects of everyday life. However, this widespread integration has sparked concerns regarding the security vulnerabilities associated with these advanced models. As a result, safeguarding the security of large language models has emerged as a critical priority in the field. The realm of attacking and defending against LLMs is constantly evolving, with new techniques and methods continuously being developed. Among these approaches, jailbreak attacks stand out as a significant threat, designed to circumvent model safety mechanisms and prompt the generation of inappropriate or malicious content. Traditional jailbreak attacks typically involve crafting inducement prompts for direct breaches; however, these tactics are often less effective against robustly filtered and highly comprehending large models. With the growing demand for real-time capabilities in LLMs, the need for continuous updates and iterations to incorporate new knowledge has become imperative. Retrieval-Augmented Generation (RAG) has emerged as an advanced technique to address the model's lack of up-to-date information by enabling it to leverage external knowledge bases. While RAG enhances the model's performance, it also opens up a new avenue for potential jailbreak attacks. In a groundbreaking study presented in this paper by Ziqiu Wang, Jun Liu, Shengkai Zhang, and Yang Yang, the concept of indirect jailbreak is introduced alongside an innovative approach called Poisoned-LangChain (PLC). This method leverages a poisoned external knowledge base to interact with large language models through LangChain technology, leading them to generate non-compliant or malicious dialogues inadvertently. The researchers conducted experiments involving six different large language models across three distinct categories of jailbreak issues. The results were striking, demonstrating that PLC successfully executed indirect jailbreak attacks under varying scenarios with impressive success rates ranging from 79.04% to 88.56%. This research sheds light on the evolving landscape of security threats faced by large language models and underscores the importance of developing robust defense mechanisms to safeguard against such sophisticated attacks in an increasingly interconnected digital world.