# An Overview of the Data-Loader Landscape: Comparative Performance Analysis

**Authors:** Iason Ofeidis, Diego Kiedanski, Leandros Tassiulas  
**arXiv:** [2209.13705](https://arxiv.org/abs/2209.13705)  
**Date:** 2022-09-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2209.13705v1/

## Key points

- Data loaders are important for improving the performance of training machine learning models
- Recent advancements in data loaders have shown promise in reducing training time and offering new features
- Dataloaders are a separate component in the Deep Learning workflow with a defined structure and features
- An open-source benchmark comparing popular data loading libraries in PyTorch has been developed
- The benchmark will be updated with new libraries and datasets as interest grows
- Remote training using a data stream over a public internet connection is viable under reasonable circumstances
- The impact of computing serving the data is highlighted, contrasting previous assumptions about locally cached datasets after download
- A novel approach to hyperparameter optimization for speed is introduced, aiming for at least an order of magnitude faster results compared to traditional approaches
- The paper provides valuable insights into dataloaders' role in enhancing training job performance
- It offers a comprehensive comparison of different dataloading libraries considering functionality, usability, and performance trade-offs

## Summary

The paper titled "An Overview of the Data-Loader Landscape: Comparative Performance Analysis" explores the importance of data loaders in improving the performance of training machine learning models. Dataloaders are responsible for moving data from storage into GPUs during training, and recent advancements have shown promise in reducing training time and offering new features like loading data from remote storage. The authors distinguish the dataloader as a separate component in the Deep Learning (DL) workflow and provide an outline of its structure and features. They also develop an open-source benchmark that compares popular data loading libraries in PyTorch. This benchmark will remain available to the community for adding new libraries and datasets as interest grows, with plans to update numerical results following major updates to any of the benchmarked libraries. Additionally, the paper demonstrates the viability of remote training by showing that it is possible to train a machine learning model using a data stream over a public internet connection under reasonable circumstances. They highlight the impact of computing serving the data, contrasting their approach with previous assumptions about locally cached datasets after download. The authors introduce a novel approach to hyperparameter optimization for speed, optimizing for processed samples over time as a proxy for total running time. This optimization is hardware-dependent and should be performed before long-running jobs, aiming to achieve at least an order of magnitude faster results compared to equivalent traditional approaches. Overall, this paper provides valuable insights into dataloaders' role in enhancing training job performance. It offers a comprehensive comparison of different dataloading libraries considering their functionality, usability, and performance trade-offs. The findings contribute to advancing research on efficient deep learning workflows and can guide practitioners in selecting appropriate dataloading strategies for their specific needs.