# A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations

**Authors:** Mang Ye, Xuankun Rong, Wenke Huang, Bo Du, Nenghai Yu, Dacheng Tao  
**arXiv:** [2502.14881](https://arxiv.org/abs/2502.14881)  
**Date:** 2025-02-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.14881v1/

## Key points

- Large Vision-Language Models (LVLMs) safety is a paramount concern for researchers and practitioners.
- The survey by Mang Ye et al. focuses on attacks, defenses, and evaluation methods related to LVLM safety.
- A unified framework is introduced to address vulnerabilities in LVLMs and strategies to mitigate them.
- The authors present a classification framework that distinguishes between inference and training phases with nuanced subcategories.
- Existing limitations in LVLM safety research are highlighted, along with future directions for fortifying model robustness.
- Safety evaluations on the LVLM Deepseek Janus-Pro offer strategic recommendations for advancing LVLM safety measures.
- The survey serves as a cornerstone for future research efforts and emphasizes the importance of security and ethical integrity in model development.
- A public repository has been established by the authors to compile and update advancements in LVLM safety: https://github.com/XuankunRong/Awesome-LVLM-Safety.

## Summary

In the rapidly evolving landscape of Large Vision-Language Models (LVLMs), ensuring their safety has become a paramount concern for researchers and practitioners alike. This comprehensive survey, authored by Mang Ye, Xuankun Rong, Wenke Huang, Bo Du, Nenghai Yu, and Dacheng Tao, delves deep into the realm of LVLM safety with a focus on attacks, defenses, and evaluation methods. The survey introduces a unified framework that intricately weaves together these critical components to provide a holistic view of the vulnerabilities inherent in LVLMs and the corresponding strategies to mitigate them. By dissecting the lifecycle of LVLMs, the authors present a classification framework that delineates between inference and training phases while offering nuanced subcategories for deeper insights. Moreover, the survey sheds light on existing limitations in LVLM safety research and outlines future directions aimed at fortifying the robustness of these models. Through a series of safety evaluations conducted on the cutting-edge LVLM known as Deepseek Janus-Pro, the authors offer strategic recommendations to advance LVLM safety measures and ensure their secure deployment in high-stakes real-world applications. This seminal work not only serves as a cornerstone for future research endeavors but also paves the way for developing models that not only push boundaries of multimodal intelligence but also adhere to stringent standards of security and ethical integrity. To further support ongoing research in this domain, the authors have established a public repository aimed at continuously compiling and updating latest advancements in LVLM safety: https://github.com/XuankunRong/Awesome-LVLM-Safety. With 22 pages and 2 figures encapsulating their findings,this survey stands as an invaluable resource for researchers,
industry professionals,and policymakers seeking to navigate intricate landscape of LVLM safety with precision
and foresight.