# CLIP in Medical Imaging: A Comprehensive Survey

**Authors:** Zihao Zhao, Yuxiao Liu, Han Wu, Yonghao Li, Sheng Wang, Lin Teng, Disheng Liu, Xiang Li, Zhiming Cui, Qian Wang, Dinggang Shen  
**arXiv:** [2312.07353](https://arxiv.org/abs/2312.07353)  
**Date:** 2023-12-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.07353v1/

## Key points

- Contrastive Language-Image Pre-training (CLIP) aligns text and image data, providing semantic-rich supervision to vision models.
- CLIP has shown promise in various tasks due to its generalizability and interpretability.
- Growing interest in applying CLIP to medical imaging for aligning medical vision and language or for clinical tasks.
- Survey explores refined CLIP pre-training techniques and applications in medical imaging.
- Practical utilization of CLIP pre-trained models in clinical tasks such as classification, dense prediction, and cross-modal tasks is discussed.
- Existing limitations of CLIP in medical imaging are highlighted with proposed future research directions.
- Insights provided for researchers on leveraging CLIP capabilities in medical image analysis.
- Figures illustrating features of medical image-text pairs and hierarchical dependencies among clinical findings are included for enhanced understanding.
- Taxonomy of studies focusing on CLIP in medical imaging domain presented along with GLoRIA's global-local approach to image-text feature alignment.

## Summary

, , , , 

In the rapidly evolving field of medical imaging, Contrastive Language-Image Pre-training (CLIP) has emerged as a powerful tool for aligning text and image data, offering semantic-rich supervision to vision models. This pre-training paradigm has shown great promise in various tasks due to its generalizability and interpretability. Recently, there has been a growing interest in applying CLIP to the medical imaging domain, either as a pre-training paradigm for aligning medical vision and language or as a key component for clinical tasks. This survey aims to provide a comprehensive exploration of the CLIP paradigm within the realm of medical imaging. It delves into both refined CLIP pre-training techniques and applications driven by CLIP. The survey begins with an introduction to the fundamentals of CLIP methodology before delving into how CLIP pre-training can be optimized for medical images and reports. The practical utilization of CLIP pre-trained models in various clinical tasks such as classification, dense prediction, and cross-modal tasks is also explored. The survey highlights existing limitations of CLIP in the context of medical imaging and proposes forward-looking directions to address the specific demands of this domain. Additionally, the survey discusses new trends, raises important questions, and proposes future research directions to further explore the potential implications of CLIP in medical imaging. The paper provides insights for researchers in the field of medical image analysis by offering a holistic understanding of the CLIP paradigm. Furthermore, figures illustrating fine-grained features of medical image-text pairs and hierarchical dependencies among clinical findings in chest X-rays are included to enhance understanding. The taxonomy of studies focusing on CLIP in the medical imaging domain is presented along with an overview of GLoRIA's global-local approach to image-text feature alignment. Overall, this comprehensive review serves as a valuable resource for researchers looking to leverage the capabilities of CLIP in the field of medical imaging. It offers timely insights into this rapidly evolving area and provides a multi-level taxonomy to cater to different research needs.