# Building a continuous benchmarking ecosystem in bioinformatics

**Authors:** Izaskun Mallona, Almut Luetge, Charlotte Soneson, Ben Carrillo, Reto Gerber, Daniel Incicau, Anthony Sonrel, Mark D. Robinson  
**arXiv:** [2409.15472](https://arxiv.org/abs/2409.15472)  
**Date:** 2024-09-23  
**Summary page:** https://www.summarizepaper.com/fr/paper/2409.15472v1/

## Abstract

Benchmarking, which involves collecting reference datasets and demonstrating method performances, is a requirement for the development of new computational tools, but also becomes a domain of its own to achieve neutral comparisons of methods. Although a lot has been written about how to design and conduct benchmark studies, this Perspective sheds light on a wish list for a computational platform to orchestrate benchmark studies. We discuss various ideas for organizing reproducible software environments, formally defining benchmarks, orchestrating standardized workflows, and how they interface with computing infrastructure.