# Self-Consistency Improves Chain of Thought Reasoning in Language Models

**Authors:** Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, Denny Zhou  
**arXiv:** [2203.11171](https://arxiv.org/abs/2203.11171)  
**Date:** 2022-03-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.11171v4/

## Key points

- Introduction of self-consistency decoding strategy to enhance chain-of-thought prompting in complex reasoning tasks
- Self-consistency method simulates diverse human thinking by sampling multiple reasoning paths from language models
- Demonstrated improvement in accuracy across arithmetic and commonsense reasoning benchmarks with self-consistency
- Benefits of self-consistency include aiding in collecting rationales, providing better uncertainty estimates, and improving calibration of language model outputs
- Use of a small number of paths (e.g., 5 or 10) can yield substantial gains without significant overhead
- Potential for leveraging self-consistency to generate better supervised data for model fine-tuning and more accurate predictions with fewer inference runs
- Inclusion of various language models in experiments, including UL2 and GPT-3, with detailed information on result reproduction using publicly available resources
- Ethical considerations raised regarding biases or inaccuracies in language model outputs and the importance of ongoing efforts to improve model factuality and safety

## Summary

In this paper, the authors introduce a novel decoding strategy called self-consistency to enhance the performance of chain-of-thought prompting in complex reasoning tasks. The self-consistency method aims to simulate the diverse ways in which humans think by sampling multiple reasoning paths from language models and selecting the most consistent answer among them. This approach acknowledges that there are often multiple valid ways to arrive at a correct solution in complex reasoning problems. The study demonstrates that self-consistency significantly improves accuracy across various arithmetic and commonsense reasoning benchmarks when applied to different large language models. Not only does self-consistency boost performance, but it also aids in collecting rationales during reasoning tasks and provides better uncertainty estimates and calibration of language model outputs. While self-consistency may require additional computation cost due to sampling multiple paths, the authors suggest that using a small number of paths (e.g., 5 or 10) can still yield substantial gains without significant overhead. Future work could explore leveraging self-consistency to generate better supervised data for model fine-tuning, leading to more accurate predictions with fewer inference runs. The inclusion of four different language models with varying scales in the experiments, including public models like UL2 and GPT-3, is highlighted. The authors provide detailed information on how others can reproduce their results using publicly available resources. Additionally, ethical considerations are raised regarding potential biases or inaccuracies in language model outputs, emphasizing the need for caution when interpreting results and ongoing efforts to improve model factuality and safety for real-world applications. Overall, this paper presents a compelling argument for incorporating self-consistency into chain-of-thought prompting for improved performance on complex reasoning tasks while also addressing important considerations around reproducibility and ethics in utilizing language models for decision-making processes.