# COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability

**Authors:** Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, Bin Hu  
**arXiv:** [2402.08679](https://arxiv.org/abs/2402.08679)  
**Date:** 2024-02-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.08679v1/

## Key points

- Growing focus on jailbreaks on Large Language Models (LLMs)
- Importance of considering jailbreaks with various attributes such as contextual coherence and sentiment/stylistic variations
- Study of controllable jailbreaking is highly beneficial
- Formal formulation of the problem of generating controllable attacks and its connection to controllable text generation in natural language processing
- Adaptation of Energy-based Constrained Decoding with Langevin Dynamics (COLD) algorithm for controllable text generation
- Introduction of COLD-Attack framework that automates and unifies the search for adversarial LLM attacks while considering control requirements such as fluency, stealthiness, sentiment, and left-right coherence
- Extensive experiments conducted on different LLMs demonstrating broad applicability of the framework
- Strong controllability, high success rates, and transferability across different models observed in generating attacks using the framework
- Contribution to understanding and mitigating risks associated with jailbreaks on LLMs
- Improved control over attack generation while considering diverse attributes of contextual coherence and sentiment/stylistic variations
- Availability of code for easy adoption and implementation.

## Summary

In recent times, there has been a growing focus on jailbreaks on Large Language Models (LLMs). To ensure a comprehensive evaluation of LLM safety, it is crucial to consider jailbreaks with various attributes such as contextual coherence and sentiment/stylistic variations. Therefore, studying controllable jailbreaking becomes highly beneficial. enabled by the  leads to diverse new jailbreak scenarios beyond generating fluent suffix attacks. It allows for addressing new controllable attack settings like revising a user query adversarially with minimal paraphrasing or inserting stealthy attacks in context with left-right coherence. The authors formally formulate the problem of generating controllable attacks and establish a novel connection between this problem and controllable text generation in the field of natural language processing. They adapt the Energy-based Constrained Decoding with Langevin Dynamics (COLD), an advanced and efficient algorithm in controllable text generation. The , introduced in this paper titled "COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability," automates and unifies the search for adversarial LLM attacks while considering various control requirements such as fluency, stealthiness, sentiment, and left-right coherence. Extensive experiments conducted on different LLMs including Llama-2, Mistral, Vicuna, Guanaco, and GPT-3.5 demonstrate the broad applicability of . The framework exhibits strong controllability, high success rates in generating attacks, and transferability across different models. Overall,presents a significant contribution to understanding and mitigating risks associated with jailbreaks on LLMs. The proposed  offers improved control over attack generation while considering diverse attributes of contextual coherence and sentiment/stylistic variations. The availability of code further facilitates the adoption and implementation of this framework.