# Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models

**Authors:** Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Ji-Rong Wen  
**arXiv:** [2403.09792](https://arxiv.org/abs/2403.09792)  
**Date:** 2024-03-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.09792v3/

## Key points

- Authors Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen focus on alignment in multimodal large language models (MLLMs)
- Uncover a vulnerability in the alignment process when image inputs are used
- Introduce a jailbreak technique called HADES to conceal and amplify malicious intent within text inputs using images
- Experimental findings show HADES has an impressive average Attack Success Rate (ASR) of 90.26% for LLaVA-1.5 and 71.60% for Gemini Pro Vision
- Code and data accessible through https://github.com/RUCAIBox/HADES
- Research contributes valuable insights into vulnerabilities associated with image inputs in MLLMs and offers a cutting-edge solution for addressing alignment challenges in AI systems

## Summary

In their paper titled "Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models," authors Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen delve into the issue of alignment in multimodal large language models (MLLMs). Through a systematic empirical analysis of representative MLLMs, they uncover a vulnerability in the alignment process when image inputs are utilized. Building upon this discovery, the researchers introduce a groundbreaking jailbreak technique called HADES. This method strategically conceals and amplifies malicious intent within text inputs by incorporating intricately designed images. The experimental findings demonstrate the effectiveness of HADES in circumventing existing MLLMs with an impressive average Attack Success Rate (ASR) of 90.26% for LLaVA-1.5 and 71.60% for Gemini Pro Vision when employing this novel approach. The authors provide further insights into their work by making both code and data accessible through https://github.com/RUCAIBox/HADES. This study not only sheds light on vulnerabilities associated with image inputs in MLLMs but also introduces a cutting-edge solution that significantly enhances understanding and mitigation of alignment issues in these complex language models. With their innovative methodology and compelling results, Li et al. 's research makes a valuable contribution to the field of multimodal large language models and sets a new standard for addressing alignment challenges in AI systems.