# ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline

**Authors:** Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Wenyi Zhao, Jie Tang, Yuxiao Dong  
**arXiv:** [2404.02893](https://arxiv.org/abs/2404.02893)  
**Date:** 2024-04-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.02893v1/

## Key points

- Study focuses on enhancing mathematical problem-solving capabilities and language abilities of large language models (LLMs)
- Introduces Self-Critique pipeline for feedback learning challenges in LLM alignment
- Training Math-Critique model from LLM to provide feedback signals on generated mathematical responses
- Utilizes rejective fine-tuning and direct preference optimization to improve problem-solving and language capabilities simultaneously
- Experiment results show significant enhancement in LLM's math problem-solving skills and language ability compared to larger LLMs
- Techniques developed deployed in ChatGLM online serving system with evaluation datasets and scripts available for further exploration
- Discussion on existing approaches for math problem-solving in LLMs, including prompting methods, supervised fine-tuning, reinforcement learning techniques, decoding strategies, and external tool utilization
- Importance of mathematical evaluation through benchmark datasets like GSM8k and MATH highlighted
- Detailed discussion on datasets for evaluating mathematical capabilities across different languages such as AQuA, Mathematics, SAT-Math, NumGLUE with specific mention of Chinese datasets like Math23K and CMath covering various proficiency levels

## Summary

This study focuses on enhancing the mathematical problem-solving capabilities of large language models (LLMs) while also improving their language abilities. The researchers introduce a novel approach called the Self-Critique pipeline, specifically designed to address feedback learning challenges in LLM alignment. This involves training a Math-Critique model from the LLM itself to provide feedback signals on generated mathematical responses. Rejective fine-tuning and direct preference optimization are then employed to improve both problem-solving and language capabilities simultaneously. Through experiments on academic and challenging datasets like MathUserEval using ChatGLM3-32B as the base model, it is shown that this approach significantly enhances the LLM's mathematical problem-solving skills while also improving its language ability by up to two times compared to larger LLMs. Additionally, related techniques developed in this work have been deployed in ChatGLM, an online serving LLM system. The researchers have made available evaluation datasets and scripts for further exploration. Furthermore, various existing approaches for math problem-solving in LLMs are discussed, including prompting methods, supervised fine-tuning, reinforcement learning techniques, decoding strategies, and external tool utilization. The importance of mathematical evaluation through benchmark datasets like GSM8k and MATH is highlighted in assessing cognitive reasoning abilities of LLMs. Moreover, there is a detailed discussion on available datasets for evaluating mathematical capabilities across different languages such as AQuA, Mathematics, SAT-Math, NumGLUE among others with specific mention of Chinese datasets like Math23K and CMath covering various proficiency levels from elementary school to exam-level challenges. Overall,this work provides valuable insights into advancing both language understanding and mathematical problem-solving skills in large language models through innovative methodologies and thorough experimentation.