# RoentGen: Vision-Language Foundation Model for Chest X-ray Generation

**Authors:** Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier Van der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P. Langlotz, Akshay Chaudhari  
**arXiv:** [2211.12737](https://arxiv.org/abs/2211.12737)  
**Date:** 2022-11-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.12737v1/

## Key points

- Authors address challenges of adapting multimodal models trained on natural image-text pairs to the medical domain
- Proposal to overcome distributional shift by adapting a pre-trained latent diffusion model on publicly available chest x-rays and radiology reports
- Emphasis on need for generative imaging models that accurately represent medical concepts using domain-specific vocabulary
- RoentGen model demonstrates capability to create visually convincing and diverse synthetic CXR images with control over output through radiology-specific language prompts
- Fine-tuning the model leads to significant improvements in classifier performance when trained jointly on synthetic and real images or solely on synthetic data

## Summary

, , , , 


In the paper titled "RoentGen: Vision-Language Foundation Model for Chest X-ray Generation," authors Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier Van der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P. Langlotz, and Akshay Chaudhari address the challenges of adapting multimodal models trained on natural image-text pairs to the medical domain. They highlight the differences between natural images and medical imaging data and propose a strategy to overcome this distributional shift by adapting a pre-trained latent diffusion model on publicly available chest x-rays (CXR) and their corresponding radiology reports. The authors emphasize the need for generative imaging models that accurately represent medical concepts using a domain-specific vocabulary. Their resulting model, named RoentGen, demonstrates the capability to create visually convincing and diverse synthetic CXR images while allowing control over output through radiology-specific language prompts. By fine-tuning this model on a fixed training set and utilizing it as a data augmentation method, they observe significant improvements in classifier performance when trained jointly on synthetic and real images or solely on synthetic data. Furthermore, the authors note that fine-tuning enhances the text-encoder's representation capabilities for specific diseases like pneumothorax by 25%, showcasing the potential of their approach to distill in-domain knowledge and improve overall model performance in generating high-quality medical imaging data. This work contributes valuable insights into bridging the gap between natural image-text pair models and medical imaging applications through innovative adaptation strategies and evaluation methodologies.