# SATBench: Benchmarking the speed-accuracy tradeoff in object recognition by humans and dynamic neural networks

**Authors:** Ajay Subramanian, Sara Price, Omkar Kumbhar, Elena Sizikova, Najib J. Majaj, Denis G. Pelli  
**arXiv:** [2206.08427](https://arxiv.org/abs/2206.08427)  
**Date:** 2022-06-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.08427v1/

## Key points

- Object recognition is crucial for everyday tasks like reading and driving
- Modeling this skill has been limited by the challenge of incorporating time into the process
- Humans exhibit a tradeoff between speed and accuracy in recognizing objects that is not fully understood
- Deep neural networks have shown promise but their inability to model the speed-accuracy tradeoff limits their usefulness as computational models for human object recognition
- SATBench is the first large-scale dataset of the speed-accuracy tradeoff in recognizing ImageNet images, including 148 observers and four dynamic neural networks across eight tasks
- Human accuracy increases with reaction time, as observed through testing many beep latencies or reaction times
- Cascaded dynamic neural networks are promising models of human reaction time in object recognition tasks
- SATBench provides valuable insights into how humans recognize objects through their flexible tradeoff between speed and accuracy
- The dataset offers new opportunities for developing more accurate computational models that incorporate both spatial and temporal dimensions in object recognition tasks.

## Summary

The ability to recognize objects is a crucial aspect of everyday tasks such as reading and driving. However, attempts to model this skill have been limited by the challenge of incorporating time into the process. Humans exhibit a flexible tradeoff between speed and accuracy when recognizing objects, which is an essential skill that has yet to be fully understood. Deep neural networks have shown promise in predicting peak human object recognition performance and neural activity, but their inability to model the temporal dimension, specifically the speed-accuracy tradeoff (SAT), limits their usefulness as computational models for how humans recognize objects. To address this limitation, a team of researchers led by Ajay Subramanian from New York University has presented SATBench, the first large-scale dataset of the SAT in recognizing ImageNet images. The dataset includes 148 observers and four dynamic neural networks across eight tasks. In each human trial, a beep indicating the desired reaction time sounds at a fixed delay after an image is presented, and an observer's response counts only if it occurs near the time of the beep. The team tested many beep latencies or reaction times and observed that human accuracy increases with reaction time. The researchers compared human behavior with several dynamic neural networks capable of inference-time adaptive computation using FLOPs as an analog for reaction time. They compared networks with humans on curve-fit error, category-wise correlation, and curve steepness and concluded that cascaded dynamic neural networks are promising models of human reaction time in object recognition tasks. Overall, SATBench provides valuable insights into how humans recognize objects through their flexible tradeoff between speed and accuracy. The dataset also offers new opportunities for developing more accurate computational models that incorporate both spatial and temporal dimensions in object recognition tasks.