# Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B

**Authors:** Di Zhang, Jiatong Li, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, Wanli Ouyang  
**arXiv:** [2406.07394](https://arxiv.org/abs/2406.07394)  
**Date:** 2024-06-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.07394v1/

## Key points

- The MCT Self-Refine (MCTSr) algorithm integrates Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS) for complex mathematical reasoning tasks.
- MCTSr aims to enhance decision-making frameworks within LLMs by addressing challenges of accuracy and reliability in strategic and mathematical reasoning scenarios.
- The algorithm constructs a Monte Carlo search tree through iterative processes of Selection, self-refinement, self-evaluation, and Backpropagation.
- MCTSr utilizes an improved Upper Confidence Bound (UCB) formula to optimize the exploration-exploitation balance effectively.
- Extensive experiments demonstrate the efficacy of MCTSr in solving Olympiad-level mathematical problems across multiple datasets and benchmarks.
- Integration of MCTS with LLMs enhances mathematical reasoning capabilities efficiently in various domains.
- Ongoing research is necessary for further enhancements in LLM-based mathematical reasoning capabilities.

## Summary

, , , , 

The MCT Self-Refine (MCTSr) algorithm is a novel approach that integrates Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS) to enhance performance in complex mathematical reasoning tasks. The primary focus of MCTSr is to address the challenges of accuracy and reliability faced by LLMs, particularly in strategic and mathematical reasoning scenarios. By leveraging systematic exploration and heuristic self-refinement mechanisms, MCTSr aims to improve decision-making frameworks within LLMs. The algorithm constructs a Monte Carlo search tree through iterative processes of Selection, self-refinement, self-evaluation, and Backpropagation. It utilizes an improved Upper Confidence Bound (UCB) formula to optimize the exploration-exploitation balance effectively. Extensive experiments have been conducted to demonstrate the efficacy of MCTSr in solving Olympiad-level mathematical problems across multiple datasets, including GSM8K, GSM Hard, MATH, and various benchmarks such as Math Odyssey, AIME, and OlympiadBench. Furthermore, when compared to current closed-source large models on test benchmarks, MCTSr has shown the ability to enhance the mathematical reasoning capabilities of small-parameter open-source models like LLaMa-3 to a comparable level. The integration of MCTS with LLMs has proven to be a versatile solution for solving complex problems efficiently in various domains. Recent advancements in enhancing mathematical reasoning in LLMs have been highlighted by other researchers. Methods such as collective refinement among multiple LLMs and reinforcement learning approaches have significantly boosted reasoning accuracy. However, there still exist gaps in achieving human-level performance in mathematical benchmarks. To overcome limitations related to logical or numerical errors in fine-tuned LLMs without additional fine-tuning steps, incorporating MCTS has been proposed by researchers. This approach aims to refine the model's response iteratively using the self-refine capabilities and self-reward evaluation method of LLMs along with the Monte Carlo Tree Search Algorithm. Despite these advancements, challenges remain regarding the accuracy and trustworthiness of outputs produced by LLMs. In mathematical contexts where precision is crucial, addressing issues such as hallucinations that may lead to irrelevant or factually incorrect outputs is essential for improving rational processes. Techniques like Self-Refine have shown promise in alleviating these challenges but ongoing research is necessary for further enhancements in LLM-based mathematical reasoning capabilities.