# MINDSTORES: Memory-Informed Neural Decision Synthesis for Task-Oriented Reinforcement in Embodied Systems

**Authors:** Anirudh Chari, Suraj Reddy, Aditya Tiwari, Richard Lian, Brian Zhou  
**arXiv:** [2501.19318](https://arxiv.org/abs/2501.19318)  
**Date:** 2025-01-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2501.19318v1/

## Key points

- Large language models (LLMs) have shown promise as zero-shot planners in embodied AI systems.
- LLMs lack the ability to learn from experience and construct persistent mental models, limiting their effectiveness in complex open-world environments like Minecraft.
- MINDSTORES is a novel approach that addresses this limitation by introducing an experience-augmented planning framework for embodied agents.
- MINDSTORES enables agents to build and utilize mental models through natural interaction with their environment, extending zero-shot LLM planning by maintaining a database of past experiences.
- The key innovation of MINDSTORES lies in representing accumulated experiences as natural language embeddings of (state, task, plan, outcome) tuples for efficient retrieval and reasoning by LLM planners.
- Extensive experiments in the MineDojo environment have demonstrated that MINDSTORES outperforms existing memory-based LLM planners while retaining flexibility and generalization benefits.
- Zero-shot LLM planning with the DEPS framework involves components such as descriptor summarizing current state and outcomes, explainer analyzing plan failures, planner generating action sequences, and selector ranking candidate sub-goals based on completion estimates.
- MINDSTORES represents a significant advancement towards more capable embodied AI systems that can continuously learn through natural experiences by combining memory-informed decision synthesis with task-oriented reinforcement learning principles.

## Summary

In the realm of embodied AI systems, large language models (LLMs) have shown promise as zero-shot planners. However, their lack of ability to learn from experience and construct persistent mental models limits their effectiveness in complex open-world environments like Minecraft. To address this limitation, a novel approach called MINDSTORES has been introduced. MINDSTORES is an experience-augmented planning framework that enables embodied agents to build and utilize mental models through natural interaction with their environment. Drawing inspiration from how humans develop cognitive mental models, MINDSTORES extends existing zero-shot LLM planning by maintaining a database of past experiences to inform future planning iterations. The key innovation of MINDSTORES lies in representing accumulated experiences as natural language embeddings of (state, task, plan, outcome) tuples. These embeddings can be efficiently retrieved and reasoned over by an LLM planner to generate insights and guide plan refinement for novel states and tasks. Through extensive experiments conducted in the MineDojo environment – a simulation platform for agents in Minecraft offering low-level controls – it has been demonstrated that MINDSTORES outperforms existing memory-based LLM planners in learning and applying knowledge while retaining the flexibility and generalization benefits of zero-shot approaches. Furthermore, recent work has highlighted the efficacy of zero-shot LLM planning with the DEPS (Describe, Explain, Plan and Select) framework. This iterative planning process involves components such as a descriptor summarizing current state and outcomes, an explainer analyzing plan failures, a planner generating action sequences, and a selector ranking candidate sub-goals based on completion estimates. Overall,MINDSTORES represents a significant advancement towards more capable embodied AI systems that can continuously learn through natural experiences. By combining memory-informed decision synthesis with task-oriented reinforcement learning principles,MINDSTORES paves the way for enhanced adaptability and robustness in navigating complex environments for embodied agents.