# Critical-Questions-of-Thought: Steering LLM reasoning with Argumentative Querying

**Authors:** Federico Castagna, Isabel Sassoon, Simon Parsons  
**arXiv:** [2412.15177](https://arxiv.org/abs/2412.15177)  
**Date:** 2024-12-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2412.15177v1/

## Key points

- Authors Federico Castagna, Isabel Sassoon, and Simon Parsons address the challenge faced by Large Language Models (LLMs) in logical and mathematical reasoning tasks despite advancements in AI research.
- LLMs excel at identifying data patterns but struggle with generalizing and solving reasoning problems beyond their training data.
- The authors propose leveraging critical questions from argumentation theory, specifically drawing on Toulmin's model of argumentation, to enhance LLMs' ability to identify logical errors and improve performance.
- Ensuring that conclusions drawn by LLMs are valid based on accepted premises is emphasized to mirror sound argumentative procedures.
- The proposed approach involves guiding models through a reasoning pipeline to assess and correct logical mistakes before generating responses to user prompts.
- The study demonstrates improved performance compared to baseline models and Chain-of-Thought (CoT) implementations through the integration of critical questioning techniques.
- Extensive evaluation across various LLMs using MT-Bench Reasoning and Math tasks validates the effectiveness of integrating critical questioning techniques into the reasoning process of LLMs.
- This study presents a promising strategy for enhancing the reasoning capabilities of advanced language models in AI research.

## Summary

In their paper titled "Critical-Questions-of-Thought: Steering LLM reasoning with Argumentative Querying," authors Federico Castagna, Isabel Sassoon, and Simon Parsons address the persistent challenge faced by Large Language Models (LLMs) in logical and mathematical reasoning tasks despite significant advancements in AI research. They highlight that LLMs excel as data pattern identifiers but struggle when tasked with generalizing and solving reasoning problems beyond their training data. To tackle this issue, the authors propose leveraging critical questions from argumentation theory, specifically drawing on Toulmin's model of argumentation. By incorporating these critical questions into the reasoning process of LLMs, they aim to enhance their ability to identify logical errors and improve overall performance. The authors emphasize the importance of ensuring that conclusions drawn by LLMs are valid based on accepted premises, mirroring the principles of sound argumentative procedures. This approach involves guiding the models through a reasoning pipeline where they can assess and correct any logical mistakes before generating responses to user prompts. Through this method, the authors demonstrate improved performance compared to baseline models and Chain-of-Thought (CoT) implementations. To validate their proposed approach, an extensive evaluation is conducted across various LLMs using MT-Bench Reasoning and Math tasks. The results showcase the effectiveness of integrating critical questioning techniques into the reasoning process of LLMs, ultimately enhancing their ability to tackle complex logical and mathematical challenges. Overall, this study sheds light on a promising strategy for bolstering the reasoning capabilities of advanced language models in AI research.