# Learning model-based planning from scratch

**Authors:** Razvan Pascanu, Yujia Li, Oriol Vinyals, Nicolas Heess, Lars Buesing, Sebastien Racanière, David Reichert, Théophane Weber, Daan Wierstra, Peter Battaglia  
**arXiv:** [1707.06170](https://arxiv.org/abs/1707.06170)  
**Date:** 2017-07-19  
**Summary page:** https://www.summarizepaper.com/en/paper/1707.06170v1/

## Key points

- The Imagination-based Planner is the first model-based agent capable of learning to construct, evaluate, and execute plans.
- It can engage in a variable number of imagination steps by proposing imagined actions and evaluating them using its model-based imagination.
- Imagined actions and outcomes are aggregated into a "plan context," shaping future real and imagined actions.
- The agent has the flexibility to decide how to imagine by testing alternative actions, chaining sequences together, or constructing complex "imagination trees" through learned policies.
- The agent is designed to optimize for both external rewards and computational costs associated with utilizing its imagination.
- Extensive experimentation shows that this architecture successfully solves challenging continuous control problems and develops sophisticated planning strategies in maze-solving tasks.

## Summary

This innovative approach to sequential decision-making represents the first model-based agent capable of learning to construct, evaluate, and execute plans. The Imagination-based Planner can engage in a variable number of imagination steps by proposing imagined actions and evaluating them using its model-based imagination. These imagined actions and outcomes are then aggregated into a "plan context," shaping future real and imagined actions. The agent has the flexibility to decide how to imagine by testing alternative actions, chaining sequences together, or constructing complex "imagination trees" through learned policies. Moreover, this agent is designed to optimize for both external rewards and computational costs associated with utilizing its imagination. Through extensive experimentation, the authors demonstrate that their architecture successfully solves challenging continuous control problems and develops sophisticated planning strategies in maze-solving tasks. Overall, this work represents a significant advancement in the field of model-based planning systems by providing insights into how agents can effectively learn to utilize components such as imagination for improved decision-making processes.