# Anomaly Detection by Adapting a pre-trained Vision Language Model

**Authors:** Yuxuan Cai, Xinwei He, Dingkang Liang, Ao Tong, Xiang Bai  
**arXiv:** [2403.09493](https://arxiv.org/abs/2403.09493)  
**Date:** 2024-03-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.09493v1/

## Key points

- Recent advancements in large vision and language models have shown efficacy in anomaly detection across various tasks.
- The CLIP-ADA framework is introduced for Anomaly Detection by Adapting a pre-trained CLIP model, incorporating two key enhancements:
- Introduction of a learnable prompt linked with abnormal patterns through self-supervised learning for consistent anomaly detection.
- Proposal of an anomaly region refinement strategy to improve localization accuracy and fully utilize CLIP's representation capabilities.
- During testing, anomalies are pinpointed by assessing similarity between the representation of the learnable prompt and the image.
- Extensive experiments demonstrate superior performance of CLIP-ADA, achieving state-of-the-art results on MVTec-AD and VisA datasets for both anomaly detection and localization tasks.
- The method shows promising performance even with limited training data, showcasing robustness in challenging scenarios.

## Summary

, , , , 

In the realm of anomaly detection, recent advancements in large vision and language models have showcased their efficacy across various downstream tasks. In this study, a novel unified framework dubbed CLIP-ADA is introduced for Anomaly Detection by Adapting a pre-trained CLIP model. This framework incorporates two key enhancements to enhance anomaly detection performance. Firstly, to achieve consistent anomaly detection across diverse industrial image categories, a learnable prompt is introduced and linked with abnormal patterns through self-supervised learning. Secondly, to fully harness the representation capabilities of CLIP, an anomaly region refinement strategy is proposed to improve localization accuracy. During testing, anomalies are pinpointed by directly assessing the similarity between the representation of the learnable prompt and the image. Extensive experiments conducted demonstrate the superior performance of the CLIP-ADA framework, achieving state-of-the-art results with 97.5/55.6 and 89.3/33.1 on MVTec-AD and VisA datasets for both anomaly detection and localization tasks respectively. Notably, this method also exhibits promising performance even with limited training data, showcasing its robustness in challenging scenarios. Authored by Yuxuan Cai, Xinwei He, Dingkang Liang, Ao Tong, and Xiang Bai, this paper titled "Anomaly Detection by Adapting a pre-trained Vision Language Model" presents a significant contribution to the field of anomaly detection through innovative adaptations of pre-trained models for enhanced performance in industrial image analysis applications.