# RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation

**Authors:** Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, Yitao Liang  
**arXiv:** [2403.05313](https://arxiv.org/abs/2403.05313)  
**Date:** 2024-03-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.05313v1/

## Key points

- Introduction of Retrieval-Augmented Thoughts (RAT) method to enhance reasoning and generation abilities of large language models in long-horizon tasks
- Leveraging iterative refinement of retrieval queries based on evolving reasoning thoughts for more accurate and efficient context generation
- Case analysis focusing on embodied planning in Minecraft and open-ended creative writing tasks
- RAT addressing inaccuracies in procedural steps by continuously refining thoughts with targeted retrieval, improving planning effectiveness
- Outperformance of RAT over other retrieval strategies in creative writing tasks like summarizing historical events by aligning closely with task progression and retrieving accurate information
- Implementation of rigorous pre-processing methodology to ensure validity of results and mitigate benchmark contamination
- Comprehensive evaluation demonstrating consistent outperformance of RAT over other methods in various tasks across different domains

## Summary

, , , , 

A novel method called Retrieval-Augmented Thoughts (RAT) has been introduced by researchers to enhance the reasoning and generation abilities of large language models in long-horizon tasks. This approach leverages iterative refinement of retrieval queries based on evolving reasoning thoughts, leading to more accurate and efficient context generation. The case analysis focused on two specific tasks: embodied planning in Minecraft and open-ended creative writing. In the Minecraft task, traditional methods like ChatGPT showed inaccuracies in procedural steps due to fragmented knowledge sources. However, RAT addressed this issue by continuously refining thoughts with targeted retrieval, improving planning effectiveness by ensuring a comprehensive understanding of all items involved in a plan. For creative writing tasks like summarizing historical events, RAT outperformed other retrieval strategies by aligning closely with task progression and retrieving accurate information. To ensure the validity of their results, the researchers implemented a rigorous pre-processing methodology to mitigate any potential benchmark contamination. The comprehensive evaluation across multiple benchmarks consistently demonstrated that RAT outperformed other methods in various tasks. These findings highlight the effectiveness of RAT in eliciting context-aware reasoning and improving performance in long-horizon generation tasks across different domains.