# Jailbreaking Proprietary Large Language Models using Word Substitution Cipher

**Authors:** Divij Handa, Advait Chirmule, Bimal Gajera, Chitta Baral  
**arXiv:** [2402.10601](https://arxiv.org/abs/2402.10601)  
**Date:** 2024-02-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.10601v1/

## Key points

- Authors Divij Handa, Advait Chirmule, Bimal Gajera, and Chitta Baral explore vulnerabilities in Large Language Models (LLMs) despite ethical alignment.
- Introduction of Jailbreak prompts as a threat bypassing alignment processes for LLMs.
- Pilot investigation on GPT-4 decoding safe sentences with word substitution cipher shows effectiveness.
- Crafting jailbreaking prompts by mapping unsafe words to safe alternatives results in up to 59.42% attack success rate on models like ChatGPT, GPT-4, and Gemini-Pro.
- Emphasis on the need for continued research to enhance model robustness while preserving decoding capabilities.
- Importance of responsible disclosure practices to strengthen language models against evolving attack strategies.
- Evaluation focuses on proprietary models ChatGPT, GPT4 from OpenAI and Gemini-Pro from Google using ADVBENCH dataset.

## Summary

In their paper titled "Jailbreaking Proprietary Large Language Models using Word Substitution Cipher," authors Divij Handa, Advait Chirmule, Bimal Gajera, and Chitta Baral delve into the vulnerabilities of Large Language Models (LLMs) despite being aligned with moral and ethical guidelines. They introduce the concept of Jailbreak prompts, which can bypass alignment processes and pose a threat to these models. The study begins with a pilot investigation on GPT-4, a state-of-the-art LLM, decoding safe sentences encrypted with various cryptographic methods. The results highlight the effectiveness of a simple word substitution cipher in decoding encrypted content. Building on this finding, the authors employ this encoding technique to craft jailbreaking prompts by mapping unsafe words to safe alternatives. Experimental results demonstrate an attack success rate of up to 59.42% on proprietary models like ChatGPT, GPT-4, and Gemini-Pro. Furthermore, the paper discusses the over-defensiveness of these models and emphasizes the need for continued research to enhance their robustness while preserving decoding capabilities. By proactively exploring potential avenues for exploitation and sharing findings with companies like OpenAI and Google through responsible disclosure practices, researchers can contribute to strengthening language models against evolving attack strategies. The evaluation focuses on three proprietary models - ChatGPT and GPT4 from OpenAI and Gemini-Pro from Google - emphasizing their alignment efforts and testing procedures. The assessment is conducted using the ADVBENCH dataset, with additional analysis provided in Appendix B. Overall, this work sheds light on the importance of addressing vulnerabilities in LLMs to ensure their resilience in an ever-changing landscape of AI security threats.