# Parameter-free Online Test-time Adaptation

**Authors:** Malik Boudiaf, Romain Mueller, Ismail Ben Ayed, Luca Bertinetto  
**arXiv:** [2201.05718](https://arxiv.org/abs/2201.05718)  
**Date:** 2022-01-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.05718v1/

## Key points

- Training state-of-the-art vision models is expensive for researchers and practitioners
- Adapting these models to downstream scenarios is important
- Online test-time adaptation is a practical approach when training data is inaccessible and no labelled data from the test distribution is available
- Existing methods for test-time adaptation perform well only in narrowly-defined experimental setups and sometimes fail catastrophically when their hyperparameters are not selected for the same scenario in which they are being tested
- The authors propose a conservative approach called Laplacian Adjusted Maximum-likelihood Estimation (LAME) objective, which adapts the model's output (not its parameters) and exhibits much higher average accuracy across scenarios than existing methods while being notably faster and having a much lower memory footprint
- Many real-world applications require performing adaptation online with limited data while receiving highly correlated non i.i.d data streams such as those encountered by autonomous vehicles or drones equipped with vision models
- The proposed method addresses this problem specification under the fully test time adaptation paradigm studied in recent years.
- Designing unsupervised test time adaptation systems that can operate online on potentially non iid data without assuming any knowledge of training data or procedures while not being tailored to specific models is important so that progress made by the community can be directly harnessed.

## Summary

The cost of training state-of-the-art vision models has become prohibitively expensive for researchers and practitioners. To address this issue, it is important to focus on adapting these models to a variety of downstream scenarios. One practical approach is online test-time adaptation, where training data is inaccessible, no labelled data from the test distribution is available, and adaptation can only happen at test time and on a handful of samples. In this paper, the authors investigate how test-time adaptation methods perform for a number of pre-trained models in various real-world scenarios. They show that existing methods perform well only in narrowly-defined experimental setups and sometimes fail catastrophically when their hyperparameters are not selected for the same scenario in which they are being tested. Motivated by the inherent uncertainty around the conditions that will ultimately be encountered at test time, the authors propose a conservative approach called Laplacian Adjusted Maximum-likelihood Estimation (LAME) objective. This approach adapts the model's output (not its parameters), and solves their objective with an efficient concave-convex procedure. The LAME objective exhibits much higher average accuracy across scenarios than existing methods while being notably faster and having a much lower memory footprint. The authors highlight that many real-world applications require performing adaptation online with limited data while receiving highly correlated non i.i.d data streams such as those encountered by autonomous vehicles or drones equipped with vision models. The proposed method addresses this problem specification under the fully test time adaptation paradigm studied in recent years. Overall, this paper emphasizes the importance of designing unsupervised test time adaptation systems that can operate online on potentially non iid data without assuming any knowledge of training data or procedures while not being tailored to specific models so that progress made by the community can be directly harnessed.