Example table: reasoning in LLMs

Create a free account to copy these papers Log in

Papers

Chain of Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, Denny Zhou · arXiv: 2201.11903v1 · 2022 · Advances in Neural Information Processing Systems 35

Publication found
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Denny Zhou · arXiv: 2203.11171v1 · 2022

Preprint found; no journal publication confirmed
STaR: Bootstrapping Reasoning With Reasoning
Eric Zelikman, Yuhuai Wu, Noah D. Goodman · arXiv: 2203.14465v1 · 2022
Let's Verify Step by Step
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman · arXiv: 2305.20050v1 · 2023

Preprint found; no journal publication confirmed
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu · arXiv: 2501.12948v1 · 2025 · Nature

Publication found

Comparison table

23 of 23 filled cells are backed by a passage of the paper.

Paper	Question	Method	Data	Main results	Limitations
1 Chain of Thought Prompting Elicits Reasoning in Large Language Models	Investigate whether prompting language models to generate intermediate reasoning steps improves performance on reasoning tasks. Source · Abstract In this paper, we explore chain of thought prompting as a method for improving the ability of language models to perform reasoning tasks.	Use chain-of-thought prompting with off-the-shelf language models, without fine-tuning. Source · Abstract First, we provide the exact input prompts for all tasks in Table 19–Table 26 in Appendix C (and emphasize that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).	Experiments cover arithmetic, symbolic, and commonsense reasoning tasks. The supplied excerpt does not specify dataset sizes. Source · Abstract Through experiments on arith- metic, symbolic, and commonsense reasoning, we ﬁnd that chain of thought processing is an emergent property of model scale that allows sufﬁciently large language models to perform reasoning tasks that otherwise have ﬂat scaling curves.	Chain-of-thought prompting improves reasoning performance at sufficiently large model scales, with benefits emerging around 100 billion parameters. No benchmark accuracy figures are supplied. Source · Abstract Crucially, our experi- ments suggest that successful chain of thought prompting is an emergent property of model scale—that is, the beneﬁts of chain of thought prompting only materialize at sufﬁcient model scale (around 100B parameters).	Some evaluated large language models are not publicly available, limiting direct reproducibility. Source · Abstract As our results make use of a set of large language models that is not publicly available, we take the following actions to facilitate reproducibility.
2 Self-Consistency Improves Chain of Thought Reasoning in Language Models	Test whether aggregating answers across multiple sampled reasoning paths improves language-model reasoning accuracy. Source · Abstract We explore a simple ensemble strategy, self-consistency, that signiﬁcantly improves the reasoning accuracy of large language models.	Prompt with reasoning examples, sample diverse reasoning paths, and select the most frequent final answer by plurality or majority vote. Source · Abstract The method we propose ﬁrst prompts the language model with example chains of thought, then generates a diverse set of reasoning paths by sampling from the model’s decoder. Each reasoning path might lead to a different ﬁnal answer, so we determine the optimal answer by taking a plurality or majority vote—i.e., the most commonly occurring answer (corresponding to a majority vote in the special case of only two alternatives).	Evaluation uses arithmetic and commonsense reasoning benchmarks, including GSM8K, SVAMP, MultiArith, CommonsenseQA, and ARC. Source · Abstract For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).	Reported accuracy improvements are +10% on GSM8K, +14% on SVAMP, +24% on MultiArith, +5% on CommonsenseQA, and +4%/+5% on ARC easy/challenge. No uncertainty estimates are supplied. Source · Abstract For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).	Not stated
3 STaR: Bootstrapping Reasoning With Reasoning Abstract and our summary	Investigate whether a model can bootstrap increasingly complex reasoning using a few rationale examples and a larger dataset without rationales. Source · Abstract We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning.	Iteratively generate rationales, retry incorrect answers with the correct answer supplied, and fine-tune on rationales that produce correct answers. Source · Abstract This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat.	Training uses a small set of rationale examples and a large dataset without rationales. Exact dataset sizes are not supplied. Source · Abstract We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning.	STaR outperforms direct-answer fine-tuning on multiple datasets and matches the performance of fine-tuning a 30-times-larger model on CommonsenseQA. Exact accuracies and uncertainty estimates are not supplied. Source · Abstract We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA.	Not stated
4 Let's Verify Step by Step	Compare process supervision with outcome supervision for improving the reliability of mathematical reasoning. Source · Abstract, ¶1 We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Open in the paper	Train reward models using either final-answer correctness or step-level correctness labels, then use them to score generated solutions. The generator is fixed rather than improved through reinforcement learning. Source · 2.1 Scope, ¶1 At each model scale, we use a single fixed model to generate all solutions. We call this model the generator. Open in the paper	Experiments use the MATH dataset; the released PRM800K dataset contains 800,000 human step-level feedback labels. Source · Abstract, ¶1 Our process-supervised model solves 78% of problems from a representative subset of the MATH test set. Additionally, we show that active learning significantly improves the efficacy of process supervision. To support related research, we also release PRM800K, the complete dataset of 800,000 step-level human feedback labels used to train our best reward model. Open in the paper	Process supervision outperforms outcome supervision, and the process-supervised model solves 78% of a representative MATH test subset. No uncertainty estimate is supplied. Source · Abstract, ¶1 We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set. Open in the paper	The large-scale reward-model training sets are not directly comparable, preventing a clean comparison of supervision types in that regime. Source · 2 Methods, ¶2 Unfortunately the training sets for these reward models are not directly comparable, for reasons we will discuss in Section 3. These models are therefore not ideal for making an apples-to-apples comparison of outcome and process supervision. Open in the paper
5 DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning Abstract and our summary	Investigate reasoning capabilities induced by large-scale reinforcement learning, including training without preliminary supervised fine-tuning. Source · Abstract DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities.	Train DeepSeek-R1-Zero through reinforcement learning without preliminary supervised fine-tuning. DeepSeek-R1 adds cold-start data and multi-stage training to improve readability and reasoning. Source · Abstract DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities.	DeepSeek-R1 uses cold-start data before reinforcement learning. The supplied abstract does not specify datasets or sample sizes. Source · Abstract To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates multi-stage training and cold-start data before RL.	DeepSeek-R1 achieves reasoning performance comparable to OpenAI-o1-1217. The release includes six distilled dense models ranging from 1.5 billion to 70 billion parameters; no benchmark scores or uncertainties are supplied. Source · Abstract DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.	DeepSeek-R1-Zero exhibits poor readability and language mixing. Source · Abstract However, it encounters challenges such as poor readability, and language mixing.

Where they agree

Both prompting studies report that using intermediate reasoning can improve reasoning-task performance without fine-tuning; one elicits a reasoning path, while the other aggregates answers across several sampled paths. [1, 2]
4 passages behind it

1 · Method (Abstract): First, we provide the exact input prompts for all tasks in Table 19–Table 26 in Appendix C (and emphasize that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).
1 · Main results (Abstract): Crucially, our experi- ments suggest that successful chain of thought prompting is an emergent property of model scale—that is, the beneﬁts of chain of thought prompting only materialize at sufﬁcient model scale (around 100B parameters).
2 · Method (Abstract): The method we propose ﬁrst prompts the language model with example chains of thought, then generates a diverse set of reasoning paths by sampling from the model’s decoder. Each reasoning path might lead to a different ﬁnal answer, so we determine the optimal answer by taking a plurality or majority vote—i.e., the most commonly occurring answer (corresponding to a majority vote in the special case of only two alternatives).
2 · Main results (Abstract): For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).
Both studies find benefits from training with reasoning-related supervision: iterative fine-tuning on successful rationales outperforms direct-answer fine-tuning, while step-level supervision outperforms final-answer supervision for scoring mathematical solutions. [3, 4]
4 passages behind it

3 · Method (Abstract): This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat.
3 · Main results (Abstract): We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA.
4 · Method (2.1 Scope, ¶1): At each model scale, we use a single fixed model to generate all solutions. We call this model the generator.
4 · Main results (Abstract, ¶1): We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set.
All five report favorable reasoning results from their respective approaches, although they evaluate different interventions and do not establish a common ranking of methods. [1, 2, 3, 4, 5]
15 passages behind it

1 · Method (Abstract): First, we provide the exact input prompts for all tasks in Table 19–Table 26 in Appendix C (and emphasize that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).
1 · Data (Abstract): Through experiments on arith- metic, symbolic, and commonsense reasoning, we ﬁnd that chain of thought processing is an emergent property of model scale that allows sufﬁciently large language models to perform reasoning tasks that otherwise have ﬂat scaling curves.
1 · Main results (Abstract): Crucially, our experi- ments suggest that successful chain of thought prompting is an emergent property of model scale—that is, the beneﬁts of chain of thought prompting only materialize at sufﬁcient model scale (around 100B parameters).
2 · Method (Abstract): The method we propose ﬁrst prompts the language model with example chains of thought, then generates a diverse set of reasoning paths by sampling from the model’s decoder. Each reasoning path might lead to a different ﬁnal answer, so we determine the optimal answer by taking a plurality or majority vote—i.e., the most commonly occurring answer (corresponding to a majority vote in the special case of only two alternatives).
2 · Data (Abstract): For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).
2 · Main results (Abstract): For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).
3 · Method (Abstract): This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat.
3 · Data (Abstract): We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning.
3 · Main results (Abstract): We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA.
4 · Method (2.1 Scope, ¶1): At each model scale, we use a single fixed model to generate all solutions. We call this model the generator.
4 · Data (Abstract, ¶1): Our process-supervised model solves 78% of problems from a representative subset of the MATH test set. Additionally, we show that active learning significantly improves the efficacy of process supervision. To support related research, we also release PRM800K, the complete dataset of 800,000 step-level human feedback labels used to train our best reward model.
4 · Main results (Abstract, ¶1): We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set.
5 · Method (Abstract): DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities.
5 · Data (Abstract): To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates multi-stage training and cold-start data before RL.
5 · Main results (Abstract): DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.

Where they disagree

The table does not establish a direct contradiction about model scale. Prompted reasoning benefits emerge around 100 billion parameters in one study, whereas another reports matching direct-answer fine-tuning of a model 30 times larger through rationale bootstrapping. The reinforcement-learning study releases distilled models with 1.5 billion to 70 billion parameters, but gives no scores that would test the prompting study's scale finding. [1, 3, 5]
6 passages behind it

1 · Method (Abstract): First, we provide the exact input prompts for all tasks in Table 19–Table 26 in Appendix C (and emphasize that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).
1 · Main results (Abstract): Crucially, our experi- ments suggest that successful chain of thought prompting is an emergent property of model scale—that is, the beneﬁts of chain of thought prompting only materialize at sufﬁcient model scale (around 100B parameters).
3 · Method (Abstract): This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat.
3 · Main results (Abstract): We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA.
5 · Method (Abstract): DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities.
5 · Main results (Abstract): DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.
The reported numerical results measure different outcomes rather than conflicting estimates. Self-consistency reports accuracy improvements of +10% on GSM8K, +14% on SVAMP, +24% on MultiArith, +5% on CommonsenseQA, and +4% and +5% on ARC easy and challenge, respectively. Process supervision reports 78% accuracy on a representative MATH test subset. These figures cannot directly rank the approaches because the benchmarks and reported quantities differ. [2, 4]
4 passages behind it

2 · Data (Abstract): For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).
2 · Main results (Abstract): For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).
4 · Data (Abstract, ¶1): Our process-supervised model solves 78% of problems from a representative subset of the MATH test set. Additionally, we show that active learning significantly improves the efficacy of process supervision. To support related research, we also release PRM800K, the complete dataset of 800,000 step-level human feedback labels used to train our best reward model.
4 · Main results (Abstract, ¶1): We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set.
The studies differ in where they intervene: two change inference-time prompting or answer selection without fine-tuning; one fine-tunes a generator on successful rationales; one trains reward models while keeping the generator fixed; and one trains reasoning models through reinforcement learning. Their positive results therefore support different mechanisms, not interchangeable treatments. [1, 2, 3, 4, 5]
10 passages behind it

1 · Method (Abstract): First, we provide the exact input prompts for all tasks in Table 19–Table 26 in Appendix C (and emphasize that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).
1 · Main results (Abstract): Crucially, our experi- ments suggest that successful chain of thought prompting is an emergent property of model scale—that is, the beneﬁts of chain of thought prompting only materialize at sufﬁcient model scale (around 100B parameters).
2 · Method (Abstract): The method we propose ﬁrst prompts the language model with example chains of thought, then generates a diverse set of reasoning paths by sampling from the model’s decoder. Each reasoning path might lead to a different ﬁnal answer, so we determine the optimal answer by taking a plurality or majority vote—i.e., the most commonly occurring answer (corresponding to a majority vote in the special case of only two alternatives).
2 · Main results (Abstract): For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%).
3 · Method (Abstract): This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat.
3 · Main results (Abstract): We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA.
4 · Method (2.1 Scope, ¶1): At each model scale, we use a single fixed model to generate all solutions. We call this model the generator.
4 · Main results (Abstract, ¶1): We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set.
5 · Method (Abstract): DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities.
5 · Main results (Abstract): DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.

What none of them addresses

Which approach performs best when model size, training data, benchmarks, and total computational budget are held constant?
How do accuracy gains compare with the training, inference, latency, and human-labeling costs of each approach?
Do generated reasoning steps faithfully reflect the processes that produce answers, rather than merely providing plausible explanations?
How robust are these approaches to distribution shifts, adversarial inputs, and reasoning tasks beyond the evaluated arithmetic, symbolic, commonsense, and mathematical settings?
What uncertainty, run-to-run variation, or statistical significance accompanies the reported improvements?
Do combinations of prompting, self-consistency, rationale bootstrapping, process supervision, and reinforcement learning yield complementary gains under controlled comparisons?