# MDT3D: Multi-Dataset Training for LiDAR 3D Object Detection Generalization

**Authors:** Louis Soum-Fontez, Jean-Emmanuel Deschaud, François Goulette  
**arXiv:** [2308.01000](https://arxiv.org/abs/2308.01000)  
**Date:** 2023-08-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.01000v1/

## Key points

- Supervised models in 3D object detection show improved performance when trained and tested within the same domain
- Real-world scenarios often lack relevant data for fine-tuning or domain adaptation methods
- Multi-Dataset Training for 3D Object Detection (MDT3D) leverages multiple annotated source datasets to enhance model robustness
- MDT3D introduces a label mapping strategy based on coarse labels to bridge the labeling gap between datasets
- The research team behind MDT3D has made their source code and results publicly available on GitHub

## Summary

, , , , 

In the field of 3D object detection, supervised models have shown significant improvements in performance when trained and tested within the same domain. However, real-world scenarios often present challenges where data from the target domain may not be readily available for fine-tuning or domain adaptation methods. This lack of access to relevant data can lead to difficulties in generalizing 3D object detection models trained on a specific source dataset to unseen datasets with different characteristics. To address this issue, a novel approach called Multi-Dataset Training for 3D Object Detection (MDT3D) has been developed. This method leverages information from multiple annotated source datasets to enhance the robustness of 3D object detection models when deployed in new environments with varying sensor configurations. By incorporating data from diverse sources, MDT3D aims to bridge the labeling gap between datasets by introducing a new label mapping strategy based on coarse labels. Furthermore, the implementation of MDT3D involves managing a mix of datasets during training and introducing a unique cross-dataset object injection technique. This innovative approach aims to improve the generalization capabilities of 3D object detection models across different domains. The study demonstrates that MDT3D yields significant improvements in the performance of various types of 3D object detection models. The research team behind this project, consisting of Louis Soum-Fontez, Jean-Emmanuel Deschaud, and François Goulette, has made their source code and additional results publicly available on GitHub for interested parties to access and utilize. The methodology employed in this research project was based on OpenPCDet, utilizing default hyperparameters across all trainings to maintain consistency. The total number of LiDAR scans loaded during training was approximately 1 million scans, equivalent to 30 epochs for a model trained on the Waymo dataset. By standardizing hyperparameters and ensuring equal representation from each dataset used in training, the study aimed to evaluate the impact of leveraging multiple datasets rather than simply increasing the volume of data. Overall, MDT3D represents a promising advancement in addressing domain adaptation challenges in 3D object detection tasks by harnessing insights from diverse annotated datasets and implementing innovative training strategies for improved model generalization across different environments and sensor configurations.