# A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification

**Authors:** Paul F. Jaeger, Carsten T. Lüth, Lukas Klein, Till J. Bungert  
**arXiv:** [2211.15259](https://arxiv.org/abs/2211.15259)  
**Date:** 2022-11-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.15259v1/

## Key points

- Major challenge in machine learning: reliable application of decision systems in real-world scenarios
- Approaches to detect erroneous predictions by assigning confidence scores
- Various methods for obtaining confidence scores: quantifying predictive uncertainty, learning explicit scoring functions, assessing alignment between input data and training distribution
- Fragmentation of approaches and evaluation protocols leads to inconsistencies and pitfalls in evaluating failure detection methods
- Study uncovers current pitfalls caused by inconsistent evaluation practices and proposes requirements for holistic and realistic evaluation of failure detection
- Large-scale empirical study enables benchmarking of confidence scoring functions across all relevant methods and failure sources
- Simple softmax response baseline performs best overall among evaluated methods
- Importance of reflection on evaluation practices for failure detection in image classification
- Code and trained models provided on GitHub (https://github.com/IML-DKFZ/fd-shifts)
- Paper accepted for presentation at ICLR 2023 as an oral presentation
- Authors: Paul F. Jaeger, Carsten T. Lüth, Lukas Klein, Till J. Bungert

## Summary

In the field of machine learning, one of the major challenges is the reliable application of decision systems in real-world scenarios. To address this challenge, many approaches have been developed to detect erroneous predictions by assigning confidence scores. These confidence scores can be obtained through various methods such as quantifying predictive uncertainty, learning explicit scoring functions, or assessing the alignment between input data and the training distribution. However, despite their shared goal of detecting failures in real-life applications, these approaches are currently treated as separate research fields with individual evaluation protocols. This fragmentation leads to inconsistencies and pitfalls in evaluating failure detection methods. Some evaluation protocols exclude relevant methods, while others ignore important failure sources. To address these issues, this study systematically uncovers the current pitfalls caused by inconsistent evaluation practices and proposes requirements for a holistic and realistic evaluation of failure detection. To demonstrate the importance of a unified perspective, the researchers present a large-scale empirical study that enables benchmarking of confidence scoring functions across all relevant methods and failure sources. Interestingly, the study reveals that a simple softmax response baseline performs best overall among the evaluated methods. This finding highlights the significant shortcomings in current evaluation practices within the abundance of published research on confidence scoring. The authors provide code and trained models for further exploration on GitHub (https://github.com/IML-DKFZ/fd-shifts). The paper has been accepted for presentation at ICLR 2023 as an oral presentation. The authors of this work are Paul F. Jaeger, Carsten T. Lüth, Lukas Klein, and Till J. Bungert. Overall, this study emphasizes the need for reflection on evaluation practices for failure detection in image classification and provides valuable insights into improving the reliability of machine learning-based decision systems in real-world applications by enabling benchmarking across all relevant methods and failure sources with a unified perspective that takes into account potential shortcomings in current evaluation practices within published research on confidence scoring .