CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
Authors: Jie Liu, Yixiao Zhang, Jie-Neng Chen, Junfei Xiao, Yongyi Lu, Bennett A. Landman, Yixuan Yuan, Alan Yuille, Yucheng Tang, Zongwei Zhou
AI-generated Key Points
- Public datasets are often small and partially labeled, limiting their usefulness in assessing anatomical structures and severe tumor subjects.
- The CLIP-Driven Universal Model for segmentation models has been introduced to address these limitations.
- This model uses embedding learned from Contrastive Language-Image Pre-training (CLIP) to better segment 25 organs and 6 types of tumors by exploiting the semantic relationship between abdominal structures.
- The Universal Model is developed from an assembly of 14 datasets with 3,410 CT scans and evaluated on 6,162 external CT scans from three datasets.
- It ranks first on the public leaderboard of the Medical Segmentation Decathlon (MSD) and achieves state-of-the-art results on Beyond The Cranial Vault (BTCV).
- Compared with dataset-specific models, it is computationally more efficient (six times faster), generalizes better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks.
- The design of CLIP embedding enables the Universal Model to be easily extended to new classes without catastrophically forgetting previously learned classes.
- With this breakthrough technology in organ segmentation and tumor detection, medical professionals can improve their assessments with greater accuracy and efficiency than ever before.
Abstract: An increasing number of public datasets have shown a marked clinical impact on assessing anatomical structures. However, each of the datasets is small, partially labeled, and rarely investigates severe tumor subjects. Moreover, current models are limited to segmenting specific organs/tumors, which can not be extended to novel domains and classes. To tackle these limitations, we introduce embedding learned from Contrastive Language-Image Pre-training (CLIP) to segmentation models, dubbed the CLIP-Driven Universal Model. The Universal Model can better segment 25 organs and 6 types of tumors by exploiting the semantic relationship between abdominal structures. The model is developed from an assembly of 14 datasets with 3,410 CT scans and evaluated on 6,162 external CT scans from 3 datasets. We rank first on the public leaderboard of the Medical Segmentation Decathlon (MSD) and achieve the state-of-the-art results on Beyond The Cranial Vault (BTCV). Compared with dataset-specific models, the Universal Model is computationally more efficient (6x faster), generalizes better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks. The design of CLIP embedding enables the Universal Model to be easily extended to new classes without catastrophically forgetting the previously learned classes.
Tools for this paper
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
⚠The license of the paper does not allow us to build upon its content and the AI assistant only knows about the paper metadata rather than the full article.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
⚠The license of this specific paper does not allow us to build upon its content and the summarizing tools will be run using the paper metadata rather than the full article. However, it still does a good job, and you can also try our tools on papers with more open licenses.
Similar papers summarized with our AI tools
Navigate through even more similar papers through a
tree representationLook for similar papers (in beta version)
We compare the title and abstract with papers indexed in our database. For arXiv papers, we search within the same category; for DOI and other sources, we search across sources and subjects. You can rerun the search as our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.