# NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking

**Authors:** Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, Kashyap Chitta  
**arXiv:** [2406.15349](https://arxiv.org/abs/2406.15349)  
**Date:** 2024-06-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.15349v1/

## Key points

- Benchmarking vision-based driving policies is a significant challenge in autonomous driving.
- Traditional evaluation methods lack closed-loop performance reflection or are hindered by computational demands and domain gaps between simulators and real-world data.
- NAVSIM is a novel approach that enables large-scale real-world benchmarking by unrolling bird's eye view abstractions of test scenes for short simulation horizons.
- NAVSIM decouples the evaluated policy from the environment, allowing for open-loop metric computation while maintaining alignment with closed-loop evaluations.
- The introduction of NAVSIM led to a competition at CVPR 2024, attracting 143 teams who submitted 463 entries and yielding valuable insights.
- Even simple methods like TransFuser can rival recent large-scale end-to-end driving architectures like UniAD on challenging scenarios using NAVSIM.
- The modular framework of NAVSIM allows for potential extensions with new datasets, data curation strategies, and metrics to support future challenges in autonomous driving research.

## Summary

In the field of autonomous driving, benchmarking vision-based driving policies presents a significant challenge. Traditional evaluation methods either lack closed-loop performance reflection or are hindered by computational demands and domain gaps between simulators and real-world data. To address these limitations, a novel approach called NAVSIM has been introduced as a middle ground for evaluating driving policies. By utilizing large datasets in conjunction with a non-reactive simulator, NAVSIM enables large-scale real-world benchmarking by unrolling bird's eye view abstractions of test scenes for short simulation horizons. The key innovation of NAVSIM lies in its ability to decouple the evaluated policy from the environment, allowing for open-loop metric computation while maintaining alignment with closed-loop evaluations. This approach contrasts with traditional displacement errors and has proven effective in providing more accurate assessments of driving policies. , , , , 

The introduction of NAVSIM has led to the organization of a competition at CVPR 2024, attracting 143 teams who submitted 463 entries and yielding valuable insights. Through empirical demonstrations, it has been observed that even simple methods with moderate computational requirements, such as TransFuser, can rival recent large-scale end-to-end driving architectures like UniAD on challenging scenarios. The modular framework of NAVSIM offers potential extensions with new datasets, data curation strategies, and metrics to continually support future challenges in autonomous driving research. Researchers interested in exploring this innovative evaluation paradigm can access the code for NAVSIM on GitHub at https://github.com/autonomousvision/navsim.