Summarize Paper logo SummarizePaper.com

Examples · Computer science and AI · Collections and comparison tables

Reasoning in LLMs: the papers compared side by side

From a collection of papers, the tool filled a table, one row per paper, each cell with its passage of the paper, then said where the papers agree and where they disagree.

A real example: what you get

5 papers compared on 5 columns.

  • Columns: Question, Method, Data, Main results, Limitations
  • 23 of the 23 cells with their passage of the paper
  • 3 points where they agree
  • 3 points where they disagree
  • 6 questions none of them addresses
  • Cells you correct by hand, CSV export, a chat with the papers
Where the papers disagree

The reported numerical results measure different outcomes rather than conflicting estimates. Self-consistency reports accuracy improvements of +10% on GSM8K, +14% on SVAMP, +24% on MultiArith, +5% on CommonsenseQA, and +4% and +5% on ARC easy and challenge, respectively. Process supervision reports 78% accuracy on a representative MATH test subset. These figures cannot directly rank the approaches because the benchmarks and …

Its source For arithmetic and commonsense reasoning benchmarks we find that self-consistency yields significant accuracy improvements in a variety of datasets, such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%). — Self-Consistency Improves Chain of Thought Reasoning in Language Models · Data

The first results

  1. Chain of Thought Prompting Elicits Reasoning in Large Language Models Question: Investigate whether prompting language models to generate intermediate reasoning steps improves performance on reasoning tasks.
  2. Self-Consistency Improves Chain of Thought Reasoning in Language Models Question: Test whether aggregating answers across multiple sampled reasoning paths improves language-model reasoning accuracy.
  3. STaR: Bootstrapping Reasoning With Reasoning Question: Investigate whether a model can bootstrap increasingly complex reasoning using a few rationale examples and a larger dataset without rationales.
  4. Let's Verify Step by Step Question: Compare process supervision with outcome supervision for improving the reliability of mathematical reasoning.
  5. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning Question: Investigate reasoning capabilities induced by large-scale reinforcement learning, including training without preliminary supervised fine-tuning.

See a complete example, explained step by step →

Create a free account, then Premium runs it on your own work. Continue with Google Create a free account Log in

The whole result, as its owner would share it →

Other examples · Computer science and AI

Every example →