# "Short-length" Adversarial Training Helps LLMs Defend "Long-length" Jailbreak Attacks: Theoretical and Empirical Evidence

**Authors:** Shaopeng Fu, Liang Ding, Di Wang  
**arXiv:** [2502.04204](https://arxiv.org/abs/2502.04204)  
**Date:** 2025-02-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.04204v1/

## Key points

- Jailbreak attacks against large language models (LLMs) aim to manipulate LLMs into exhibiting harmful behaviors using carefully crafted adversarial prompts.
- Authors propose adversarial training (AT)-based alignment as a strategy to defend against jailbreak attacks, involving training LLMs on adversarial prompts to learn safe responses.
- The length of adversarial prompts significantly impacts the robustness of aligned LLMs, with a focus on defending against jailbreak attacks with specific lengths of adversarial suffixes.
- Theoretical analysis involves studying adversarial in-context learning of linear transformers and establishing a generalization bound for trained transformers based on perturbed in-context samples during training and testing.
- Empirical experiments show that implementing "short-length" AT strategies can effectively defend against "long-length" jailbreak attacks targeting LLMs.
- Results indicate a positive correlation between the success rate of jailbreak attacks and the ratio between the square root of the adversarial suffix length during jailbreaking and the length during AT.

## Summary

In their paper titled "Short-length" Adversarial Training Helps LLMs Defend "Long-length" Jailbreak Attacks: Theoretical and Empirical Evidence, authors Shaopeng Fu, Liang Ding, and Di Wang delve into the realm of jailbreak attacks against large language models (LLMs). These attacks are designed to manipulate LLMs into exhibiting harmful behaviors by using carefully crafted adversarial prompts. To counter such attacks, the authors propose a strategy known as adversarial training (AT)-based alignment. This involves training LLMs on some of the most adversarial prompts to help them learn how to respond safely when under attack. A key finding of their study is the significant role that the length of adversarial prompts plays in determining the robustness of aligned LLMs. Specifically focusing on adversarial suffix jailbreak attacks, the authors reveal that defending against a jailbreak attack with an adversarial suffix of length $\Theta(M)$ can be achieved by aligning LLMs on prompts with adversarial suffixes of length $\Theta(\sqrt{M})$. The authors provide both theoretical and empirical evidence to support their claims. Theoretical analysis centers around studying the adversarial in-context learning of linear transformers on linear regression tasks. They establish a robust generalization bound for trained transformers, which depends on terms related to the number of adversarially perturbed in-context samples during training and testing. Empirically, the authors conduct AT experiments on popular open-source LLMs to evaluate their robustness against jailbreak attacks with varying lengths of adversarial suffixes. Their results demonstrate a positive correlation between the success rate of these attacks and the ratio between the square root of the adversarial suffix during jailbreaking and the length during AT. This suggests that it is feasible to defend against "long-length" jailbreak attacks by efficiently implementing "short-length" AT strategies. Overall, this research sheds light on effective defense mechanisms against jailbreak attacks targeting LLMs and provides valuable insights into enhancing their resilience in real-world scenarios. The code for this study is available at https://github.com/fshp971/adv-icl.