# BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

**Authors:** Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao Yu, Jifeng Dai  
**arXiv:** [2203.17270](https://arxiv.org/abs/2203.17270)  
**Date:** 2022-03-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.17270v2/

## Key points

- Autonomous driving systems require 3D visual perception tasks such as 3D detection and map segmentation based on multi-camera images.
- A new framework called BEVFormer has been proposed to learn unified Bird's-Eye-View (BEV) representations that support multiple autonomous driving perception tasks.
- The model employs spatial cross-attention and temporal self-attention to extract spatial features from regions of interest across camera views for each BEV query and recurrently fuse historical BEV information.
- The proposed approach achieves state-of-the-art performance with an NDS metric score of 56.9% on the nuScenes test set which is 9.0 points higher than previous best arts and comparable to LiDAR-based baselines' performance.
- It significantly improves velocity estimation accuracy and object recall under low visibility conditions, but some mistakes in small and remote objects were observed in visualization results.
- Camera-based methods still have a gap compared to LiDAR-based methods in terms of effectiveness and efficiency.
- The proposed framework has tremendous potential for further exploration in improving perception accuracy in 3D space critical for autonomous driving systems' safe operation.
- The code is available at https://github.com/zhiqi-li/BEVFormer.

## Summary

The development of autonomous driving systems requires the ability to perform 3D visual perception tasks such as 3D detection and map segmentation based on multi-camera images. To this end, a new framework called BEVFormer has been proposed which utilizes spatiotemporal transformers to learn unified Bird's-Eye-View (BEV) representations that support multiple autonomous driving perception tasks. The model employs spatial cross-attention to extract spatial features from regions of interest across camera views for each BEV query and temporal self-attention to recurrently fuse historical BEV information. The proposed approach achieves state-of-the-art performance with an NDS metric score of 56.9% on the nuScenes test set which is 9.0 points higher than previous best arts and comparable to LiDAR-based baselines' performance. Furthermore, it significantly improves velocity estimation accuracy and object recall under low visibility conditions. Visualization results show impressive outcomes except for some mistakes in small and remote objects. The limitations of camera-based methods are acknowledged as they still have a gap compared to LiDAR-based methods in terms of effectiveness and efficiency. Overall, the proposed framework has tremendous potential for further exploration in improving perception accuracy in 3D space critical for autonomous driving systems' safe operation. The code is available at https://github.com/zhiqi-li/BEVFormer.