# MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying

**Authors:** Shaoshuai Shi, Li Jiang, Dengxin Dai, Bernt Schiele  
**arXiv:** [2306.17770](https://arxiv.org/abs/2306.17770)  
**Date:** 2023-06-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.17770v1/

## Key points

- Motion prediction is crucial for autonomous driving systems to navigate complex scenarios and make informed decisions.
- The Motion TRansformer (MTR) framework utilizes transformer encoder-decoder structure with learnable intention queries for efficient and accurate future trajectory prediction.
- MTR enhances multimodal motion prediction by customizing intention queries for different motion modalities, improving efficiency, and accuracy.
- MTR++ extends the capabilities of MTR to predict multimodal motion for multiple agents simultaneously through symmetric context modeling and mutually-guided intention querying modules.
- Experimental results show that both MTR and MTR++ frameworks achieve state-of-the-art performance in motion prediction benchmarks, with MTR++ exhibiting enhanced performance and efficiency compared to its predecessor.

## Summary

, , , , 

Motion prediction is a critical component of autonomous driving systems, allowing them to navigate complex driving scenarios and make informed decisions. This task is challenging due to the varied behaviors of traffic participants and the intricate environmental contexts in which they operate. To address these challenges, the Motion TRansformer (MTR) frameworks have been proposed in this paper. The initial MTR framework leverages a transformer encoder-decoder structure with learnable intention queries, enabling efficient and accurate prediction of future trajectories. By customizing intention queries for different motion modalities, MTR enhances multimodal motion prediction while reducing reliance on dense goal candidates. The framework consists of two key processes: global intention localization, which identifies the agent's intent to improve overall efficiency, and local movement refinement, which adaptively refines predicted trajectories for enhanced accuracy. Furthermore, an advanced version of the MTR framework, known as MTR++, has been introduced in this paper. MTR++ extends the capabilities of MTR to predict multimodal motion for multiple agents simultaneously. It incorporates symmetric context modeling and mutually-guided intention querying modules to facilitate interaction among multiple agents' future behaviors, resulting in scene-compliant future trajectories. Experimental results demonstrate that the MTR framework achieves state-of-the-art performance on competitive motion prediction benchmarks. Additionally, the MTR++ framework surpasses its predecessor by exhibiting enhanced performance and efficiency in predicting accurate multimodal future trajectories for multiple agents. Moreover, detailed analyses comparing inference latency between MTR and MTR++, efficiency comparisons based on memory usage for different numbers of focal agents per scene, as well as performance comparisons are provided in this study. The findings show that not only does MTR++ better preserve input locality structure but also improves memory efficiency for larger map encodings required for long-term motion prediction. In terms of multimodal future behavior modeling within encoded scene context features, various strategies have been explored by existing works. These include generating trajectory samples to approximate output distribution and other studies focusing on generating a full trajectory for each goal scenario. Overall, this paper presents a comprehensive overview of the Motion TRansformer frameworks (MTR and MTR++) and their advancements in multi-agent motion prediction with symmetric scene modeling and guided intention querying techniques.