# CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models

**Authors:** Feiyang Li, Peng Fang, Zhan Shi, Arijit Khan, Fang Wang, Dan Feng, Weihao Wang, Xin Zhang, Yongjian Cui  
**arXiv:** [2504.13534](https://arxiv.org/abs/2504.13534)  
**Date:** 2025-04-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2504.13534v1/

## Key points

- Authors introduce CoT-RAG framework to enhance reasoning in large language models (LLMs)
- Challenges faced by existing CoT reasoning approach include reliability of LLMs and interference of natural language reasoning on LLM inference logic
- Knowledge Graph-driven CoT Generation for modulating LLMs' reasoning chain generation
- Learnable Knowledge Case-aware RAG for providing LLMs with learnable information through retrieval-augmented generation
- Pseudo-Program Prompting Execution to encourage logical rigor in reasoning tasks
- Evaluation on nine public datasets shows significant accuracy improvements ranging from 4.0% to 23.0%
- Testing on domain-specific datasets demonstrates remarkable accuracy and efficient execution, highlighting practical applicability and scalability of CoT-RAG

## Summary

, , , , 

In their paper titled "CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models," authors Feiyang Li, Peng Fang, Zhan Shi, Arijit Khan, Fang Wang, Dan Feng, Weihao Wang, Xin Zhang, and Yongjian Cui introduce a novel reasoning framework aimed at improving the performance of large language models (LLMs) in complex tasks. The existing chain-of-thought (CoT) reasoning approach faces challenges related to the reliability of LLMs in generating reasoning chains and the interference of natural language reasoning on LLM inference logic. To overcome these obstacles, the authors propose CoT-RAG with three key designs. Firstly, they implement Knowledge Graph-driven CoT Generation to leverage knowledge graphs for modulating LLMs' reasoning chain generation and enhancing credibility. Secondly, they incorporate Learnable Knowledge Case-aware RAG by integrating retrieval-augmented generation (RAG) into knowledge graphs to retrieve relevant sub-cases and sub-descriptions for providing LLMs with learnable information. Lastly, they introduce Pseudo-Program Prompting Execution to encourage LLMs to execute reasoning tasks in pseudo-programs with greater logical rigor. The authors conducted a comprehensive evaluation on nine public datasets covering three reasoning problems. Compared to state-of-the-art methods, CoT-RAG demonstrated significant accuracy improvements ranging from 4.0% to 23.0%. Additionally, testing on four domain-specific datasets showcased remarkable accuracy and efficient execution of CoT-RAG, underscoring its practical applicability and scalability in real-world scenarios. This refined summary highlights the innovative approach taken by the authors in addressing challenges faced by existing CoT reasoning frameworks and emphasizes the promising results achieved through the implementation of CoT-RAG.