# Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology

**Authors:** Richard J. Chen, Rahul G. Krishnan  
**arXiv:** [2203.00585](https://arxiv.org/abs/2203.00585)  
**Date:** 2022-03-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.00585v1/

## Key points

- Tissue phenotyping is crucial in understanding histopathologic biomarkers in cancer pathology
- Analyzing whole-slide images (WSIs) presents challenges due to high image resolutions and variability in tissue labeling
- Previous studies have suggested using pretrained image encoders for feature extraction
- Authors Richard J. Chen and Rahul G. Krishnan trained self-supervised models to identify effective representations for pathology
- Vision Transformers utilizing DINO-based knowledge distillation can learn interpretable and data-efficient features in histology images
- Different attention heads within the Vision Transformers learn distinct morphological phenotypes
- Evaluation code and pretrained weights are publicly available on GitHub for further exploration and utilization by the research community

## Summary

, , , , 

In the field of cancer pathology, tissue phenotyping plays a crucial role in understanding histopathologic biomarkers within the tumor-immune microenvironment. However, analyzing whole-slide images (WSIs) poses significant challenges. Firstly, WSIs have extremely high image resolutions, making it difficult to perform large-scale pixel-level data curation. Additionally, the diverse morphological phenotypes observed in tissues lead to variability in tissue labeling among different observers. To overcome these limitations, previous studies have suggested using pretrained image encoders that leverage transfer learning from ImageNet or self-supervised pretraining to extract morphological features from pathology. However, the effectiveness of these approaches has not been extensively validated. In this study, authors Richard J. Chen and Rahul G. Krishnan aim to identify effective representations for pathology by training various self-supervised models and evaluating them on weakly-supervised and patch-level tasks. Their key finding is that Vision Transformers utilizing DINO-based knowledge distillation can learn interpretable and data-efficient features in histology images. Notably, different attention heads within the Vision Transformers learn distinct morphological phenotypes. The authors provide evaluation code and pretrained weights publicly available on GitHub for further exploration and utilization by the research community. Overall, this research contributes to advancing our understanding of tissue phenotyping in cancer pathology by demonstrating the efficacy of self-supervised Vision Transformers in extracting meaningful features from histology images.