# Efficient Adaptive Ensembling for Image Classification

**Authors:** Antonio Bruno, Davide Moroni, Massimo Martinelli  
**arXiv:** [2206.07394](https://arxiv.org/abs/2206.07394)  
**Date:** 2022-06-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.07394v3/

## Key points

- Computer Vision field has seen a trend of minor improvements in image classification performance at the cost of increased complexity
- Authors propose a novel method to boost image classification performances without increasing complexity
- They revisit ensembling, a powerful approach often not used properly due to complexity and long training time
- Proposed method involves training two EfficientNet-b0 end-to-end models on disjoint subsets of data using bagging
- An efficient adaptive ensemble is created by fine-tuning a trainable combination layer
- Outperforms state-of-the-art by an average of 0.5% on accuracy across major benchmark datasets
- Achieves improved results while maintaining restrained complexity (reduced parameters and FLOPS)
- Additional details provided about experiments and results (no hyperparameter tuning, multiple trainings with different random seeds)
- Highest improvements observed on tasks where accuracy is below 99%
- Promising solution for improving computer vision tasks

## Summary

In recent times, the field of Computer Vision has seen a trend where minor improvements in image classification performance are achieved at the cost of significant increases in complexity. To address this issue, the authors propose a novel method to boost image classification performances without increasing complexity. They revisit ensembling, a powerful approach that is often not used properly due to its complex nature and long training time. However, they make it feasible through a specific design choice. The proposed method involves training two EfficientNet-b0 end-to-end models on disjoint subsets of data using bagging. Then, an efficient adaptive ensemble is created by performing fine-tuning of a trainable combination layer. This approach allows them to outperform the state-of-the-art by an average of 0.5% on accuracy across several major benchmark datasets. What sets their method apart is that it achieves these improved results while maintaining restrained complexity. The number of parameters is reduced by 5-60 times compared to existing methods, and the Floating point Operations Per Second (FLOPS) required is reduced by 10-100 times. These reductions in complexity make their method more efficient and practical for real-world applications. The authors also provide additional details about their experiments and results such as no hyperparameter tuning involved as all hyperparameters were prefixed except for the random seeds used for different runs; 2 end-to-end weak trainings (one for each subset) and 5 fine-tuning ensemble trainings using different random seeds; highest improvements (>0.5%) observed on tasks where accuracy is below 99%. Overall, this refined summary highlights how the authors propose a novel method to boost image classification performances without increasing complexity. Their approach involves revisiting ensembling and making it feasible through a specific design choice which leads to improved results while maintaining restrained complexity - making it a promising solution for improving computer vision tasks.