# Learning To Teach Large Language Models Logical Reasoning

**Authors:** Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang, Dongsheng Li  
**arXiv:** [2310.09158](https://arxiv.org/abs/2310.09158)  
**Date:** 2023-10-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.09158v1/

## Key points

- Authors Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang, and Dongsheng Li address challenges faced by large language models (LLMs) in logical reasoning tasks.
- LLMs often produce unreliable content in practical reasoning scenarios due to issues like hallucination.
- The authors conduct a study focusing on tasks such as event relation extraction and deductive reasoning to highlight LLMs' shortcomings in rigorous reasoning.
- LLMs struggle with tasks requiring precise logic and tend to generate counterfactual answers, necessitating iterative refinement.
- The authors explore strategies to enhance LLMs' logical reasoning skills for improved consistency across diverse contexts.
- They introduce a synthesized dataset called LLM-LR that incorporates multi-hop reasoning for evaluation and pre-training purposes.
- Extensive quantitative and qualitative analyses demonstrate the importance of incorporating logic into LLM training for better performance in various tasks.

## Summary

In their paper "Learning To Teach Large Language Models Logical Reasoning," authors Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang, and Dongsheng Li delve into the challenges faced by large language models (LLMs) in logical reasoning tasks. Despite the remarkable language generation capabilities and generalization power of LLMs, they often produce unreliable content in practical reasoning scenarios due to issues like hallucination. To address this issue, the authors conduct a thorough investigation to assess the logical reasoning abilities of LLMs. Their study focuses on various tasks such as event relation extraction and deductive reasoning to highlight the shortcomings of LLMs in rigorous reasoning. The research reveals that LLMs struggle with tasks requiring precise logic and tend to generate counterfactual answers, necessitating iterative refinement. In response to these findings, the authors explore different strategies to enhance LLMs' logical reasoning skills, aiming to improve the consistency of their answers across diverse contexts. Furthermore, the authors introduce a synthesized dataset called LLM-LR that incorporates multi-hop reasoning for evaluation and pre-training purposes. Through extensive quantitative and qualitative analyses across different tasks, they demonstrate the effectiveness and importance of incorporating logic into LLM training. This work not only sheds light on teaching LLMs logical reasoning but also provides valuable insights for leveraging these models in practical applications moving forward.