# No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems

**Authors:** Nimit S. Sohoni, Jared A. Dunnmon, Geoffrey Angus, Albert Gu, Christopher Ré  
**arXiv:** [2011.12945](https://arxiv.org/abs/2011.12945)  
**Date:** 2020-11-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2011.12945v2/

## Key points

- The paper addresses the issue of hidden stratification in real-world classification tasks
- Hidden stratification leads to variable performance across subclasses when models are trained using only coarse-grained class labels
- The proposed method, GEORGE, aims to measure and mitigate hidden stratification even without subclass labels
- GEORGE leverages clustering techniques to estimate subclass labels for training data without explicit annotations
- The authors theoretically characterize the performance of GEORGE by considering worst-case generalization error across any subclass
- Empirical evaluation shows that GEORGE significantly improves worst-case subclass accuracy compared to standard training techniques, achieving up to a 22 percentage point boost
- GEORGE provides a practical approach for improving model performance across fine-grained subclasses even without explicit subclass labels.

## Summary

, , , , 

The paper titled "No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems" addresses the issue of hidden stratification in real-world classification tasks. In such tasks, each class often consists of multiple finer-grained "subclasses," but the subclass labels are frequently unavailable. This lack of subclass information leads to highly variable performance across different subclasses when models are trained using only the coarser-grained class labels. Hidden stratification poses significant challenges for models deployed in safety-critical applications like medicine. To tackle this problem, the authors propose a method called GEORGE, which aims to measure and mitigate hidden stratification even when subclass labels are unknown. The key insight is that unlabeled subclasses can often be separated in the feature space of deep neural networks. To estimate subclass labels for training data without explicit annotations, GEORGE leverages clustering techniques. These approximate subclass labels serve as noisy supervision in a distributionally robust optimization objective. The authors theoretically characterize the performance of GEORGE by considering worst-case generalization error across any subclass. To validate their approach, they empirically evaluate GEORGE on a combination of real-world and benchmark image classification datasets. They demonstrate that their method significantly improves worst-case subclass accuracy compared to standard training techniques, achieving up to a 22 percentage point boost without requiring prior knowledge about the subclasses. Overall, this research introduces an effective solution to address hidden stratification in coarse-grained classification problems. By leveraging deep neural networks and clustering techniques, GEORGE provides a practical approach for improving model performance across fine-grained subclasses even when explicit subclass labels are unavailable.