# TD-MPC2: Scalable, Robust World Models for Continuous Control

**Authors:** Nicklas Hansen, Hao Su, Xiaolong Wang  
**arXiv:** [2310.16828](https://arxiv.org/abs/2310.16828)  
**Date:** 2023-10-25  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2310.16828v2/

## Key points

- TD-MPC2 is a model-based reinforcement learning algorithm focusing on optimizing local trajectories within the latent space of an implicit world model
- Unlike its predecessor TD-MPC, TD-MPC2 does not require decoders and has shown significant improvements across various online RL tasks in different domains
- With a single set of hyperparameters, TD-MPC2 consistently delivers strong results and demonstrates enhanced agent capabilities as both the model and data sizes increase
- A single 317M parameter agent trained using TD-MPC2 successfully performs 80 tasks spanning multiple domains, embodiments, and action spaces
- The scalability and robustness of TD-MPC2 are highlighted in handling complex real-world scenarios
- The authors emphasize democratizing RL by enhancing open-source algorithms like TD-MPC2 to make them more accessible to smaller teams and individuals with limited resources
- While other methods like DreamerV3 excel in specific domains, TD-MPC2 stands out for its versatility and performance across a broader spectrum of challenges
- Researchers and practitioners are invited to join efforts in advancing algorithmic robustness within reinforcement learning by refining existing approaches like TD-MPC2

## Summary

TD-MPC2 is a highly effective model-based reinforcement learning algorithm that focuses on optimizing local trajectories within the latent space of an implicit world model. Unlike its predecessor TD-MPC, this approach does not require decoders and has shown significant improvements across various online RL tasks in different domains. With a single set of hyperparameters, TD-MPC2 consistently delivers strong results and demonstrates enhanced agent capabilities as both the model and data sizes increase. In a groundbreaking demonstration, a single 317M parameter agent trained using TD-MPC2 successfully performs 80 tasks spanning multiple domains, embodiments, and action spaces. This remarkable achievement highlights the scalability and robustness of the algorithm in handling complex real-world scenarios. The authors emphasize the importance of democratizing RL by enhancing the robustness of open-source algorithms like TD-MPC2 to make them more accessible to smaller teams and individuals with limited resources. The development of TD-MPC2 showcases the ongoing quest for an all-encompassing RL algorithm that excels across diverse tasks without extensive tuning or expert intervention. While other contemporary methods like DreamerV3 have shown promise in specific domains such as Atari games and Minecraft, TD-MPC2 stands out for its versatility and performance across a broader spectrum of challenges. The authors invite researchers and practitioners to join their efforts in advancing algorithmic robustness within the field of reinforcement learning. By continuing to refine existing approaches like TD-MPC2 and exploring new avenues for improvement, they aim to drive impactful advancements that benefit the entire RL community. For further insights, videos, models, data, code, and more information on TD-MPC2 can be accessed at https://tdmpc2.com.