# CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability

**Authors:** Martin Mundt, Steven Lang, Quentin Delfosse, Kristian Kersting  
**arXiv:** [2110.03331](https://arxiv.org/abs/2110.03331)  
**Date:** 2021-10-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2110.03331v2/

## Key points

- Continual machine learning poses unique challenges for set-up and evaluation
- Critiques have been raised on the narrowness of algorithm-centric perspectives and evaluation protocols
- The authors introduce the Continual Learning EValuation Assessment Compass (CLEVA-Compass) to address these issues
- CLEVA-Compass provides a visual means to identify how approaches are practically reported and contextualized in the broader literature landscape
- CLEVA-Compass promotes research transparency and comparability by providing an intuitive chart that highlights the priorities of individual systems, where they resemble each other, and what elements are missing towards a fair comparison
- This approach is necessary to move beyond current limitations in evaluating continual learning algorithms and enable more comprehensive comparisons across different scenarios
- CLEVA-Compass represents an important step forward in promoting research transparency and comparability in continual machine learning.

## Summary

The field of continual machine learning poses unique challenges for set-up and evaluation. Critiques have been raised on the narrowness of algorithm-centric perspectives and evaluation protocols. To address these issues, the authors of this paper introduce the Continual Learning EValuation Assessment Compass (CLEVA-Compass). This provides a visual means to identify how approaches are practically reported and contextualized in the broader literature landscape. Rather than aiming for a precise formulation of desiderata which may not be applicable across diverse applications, CLEVA-Compass promotes research transparency and comparability by providing an intuitive chart that highlights the priorities of individual systems, where they resemble each other, and what elements are missing towards a fair comparison. The authors argue that this approach is necessary to move beyond current limitations in evaluating continual learning algorithms and enable more comprehensive comparisons across different scenarios. Overall, CLEVA-Compass represents an important step forward in promoting research transparency and comparability in continual machine learning.