# DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

**Authors:** Yue Wang, Vitor Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, Justin Solomon  
**arXiv:** [2110.06922](https://arxiv.org/abs/2110.06922)  
**Date:** 2021-10-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2110.06922v1/

## Key points

- Novel framework for multi-camera 3D object detection
- Top-down approach using sparse 3D object queries
- Outperforms bottom-up methods and eliminates post-processing techniques
- Achieves state-of-the-art performance on nuScenes benchmark
- Comparison with pseudo-LiDAR approaches showing superior results in various metrics

## Summary

, , , , 

In their paper titled "DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries," Yue Wang, Vitor Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon introduce a novel framework for multi-camera 3D object detection. The proposed method operates directly in 3D space by extracting 2D features from multiple camera images and using a sparse set of 3D object queries to index into these features. This top-down approach outperforms bottom-up methods that rely on per-pixel depth estimation and eliminates the need for post-processing techniques like non-maximum suppression, resulting in significantly improved inference speed. The authors demonstrate the effectiveness of their approach by achieving state-of-the-art performance on the nuScenes autonomous driving benchmark. They also compare their method with pseudo-LiDAR approaches commonly used for 3D object detection and show superior results in terms of metrics such as NDS (Normalized Detection Score), mAP (mean Average Precision), mATE (mean Average Translation Error), mASE (mean Average Scale Error), mAOE (mean Average Orientation Error), mAVE (mean Average Volume Error), and mAAE (mean Average Aspect Ratio Error). Furthermore, the authors implement a baseline pseudo-LiDAR method using a pre-trained PackNet network to validate that their proposed approach is more effective than explicit depth prediction methods. The study concludes by emphasizing the significance of their top-down approach in improving accuracy and efficiency in multi-camera 3D object detection tasks. Overall, this paper presents an innovative solution that utilizes sparse 3D queries to directly operate in 3D space, resulting in improved performance and faster inference speed. , , , , and  are the key concepts addressed in this paper.