# Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

**Authors:** Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, Mao Yang  
**arXiv:** [2408.06195](https://arxiv.org/abs/2408.06195)  
**Date:** 2024-08-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2408.06195v1/

## Key points

- Introduction of rStar, a self-play mutual reasoning approach for enhancing the reasoning capabilities of small language models (SLMs)
- Key innovation of rStar: decoupling reasoning into a self-play mutual generation-discrimination process
- Use of Monte Carlo Tree Search (MCTS) to enrich SLM with diverse human-like reasoning actions and construct higher quality reasoning trajectories
- Validation of generated trajectories by a discriminator SLM to ensure accuracy
- Effectiveness of rStar demonstrated through experiments on various reasoning problems such as GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA
- Remarkable improvements in accuracy levels on the GSM8K dataset for different models using rStar
- Ablation study conducted to evaluate the effectiveness of rich action space in solving complex reasoning tasks
- Highlighting how mutual reasoning through self-play can enhance problem-solving abilities of smaller language models without external sources or fine-tuning processes associated with larger models like GPT-4
- Availability of rStar code for further exploration and implementation at https://github.com/zhentingqi/rStar

## Summary

In this paper, the authors introduce rStar, a self-play mutual reasoning approach that enhances the reasoning capabilities of small language models (SLMs) without the need for fine-tuning or reliance on superior models. The key innovation of rStar lies in decoupling reasoning into a self-play mutual generation-discrimination process. This allows for a target SLM to enrich the Monte Carlo Tree Search (MCTS) with a diverse set of human-like reasoning actions and construct higher quality reasoning trajectories. Another SLM then functions as a discriminator to validate each trajectory generated by the target SLM. The mutually agreed upon trajectories are deemed mutual consistent and are thus more likely to be accurate. Extensive experiments conducted across five different SLMs showcase the effectiveness of rStar in solving various reasoning problems such as GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Notably, rStar demonstrates remarkable improvements in accuracy levels for different models on the GSM8K dataset - boosting accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for Mistral-7B, and from 74.53% to 91.13% for LLaMA3-8B-Instruct. The authors also conduct an ablation study on the effectiveness of their rich action space by evaluating LLaMA3-8B on 200 sampled GSM8K questions. This study provides insights into how different combinations of actions impact the model's performance in solving complex reasoning tasks. Overall, this research highlights how mutual reasoning through self-play can significantly enhance the problem-solving abilities of smaller language models without relying on external sources or fine-tuning processes typically associated with larger models like GPT-4. The code for rStar will be made available at https://github.com/zhentingqi/rStar for further exploration and implementation by interested researchers and practitioners in the field.