ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
Auteurs : Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Wenyi Zhao, Jie Tang, Yuxiao Dong
Résumé scientifique
Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving. While many strategies and datasets to enhance LLMs' mathematics are developed, it remains a challenge to simultaneously maintain and improve both language and mathematical capabilities in deployed LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses the challenge in the feedback learning stage of LLM alignment. We first train a general Math-Critique model from the LLM itself to provide feedback signals. Then, we sequentially employ rejective fine-tuning and direct preference optimization over the LLM's own generations for data collection. Based on ChatGLM3-32B, we conduct a series of experiments on both academic and our newly created challenging dataset, MathUserEval. Results show that our pipeline significantly enhances the LLM's mathematical problem-solving while still improving its language ability, outperforming LLMs that could be two times larger. Related techniques have been deployed to ChatGLM\footnote{\url{https://chatglm.cn}}, an online serving LLM. Related evaluation dataset and scripts are released at \url{https://github.com/THUDM/ChatGLM-Math}.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
Improving Large Language Model Fine-tuning for Solving Math ProblemsYixin Liu, Avi Singh, C. D. Freeman et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 78 fois · Pas encore résumé
Résumé scientifique
Despite their success in many natural language tasks, solving math problems remains a significant challenge for large language models (LLMs). A large gap exists between LLMs' pass-at-one and pass-at-N performance in solving math problems, suggesting LLMs might be close to finding correct solutions, motivating our exploration of fine-tuning methods to unlock LLMs' performance. Using the challenging MATH dataset, we investigate three fine-tuning strategies: (1) solution fine-tuning, where we fine-tune to generate a detailed solution for a given math problem; (2) solution-cluster re-ranking, where the LLM is fine-tuned as a solution verifier/evaluator to choose among generated candidate solutio…
-
SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language ModelsDian Yu, Baolin Peng, Ye Tian et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 5 fois · Pas encore résumé
Résumé scientifique
There is a growing trend of teaching large language models (LLMs) to solve mathematical problems through coding. Existing studies primarily focus on prompting powerful, closed-source models to generate seed training data followed by in-domain data augmentation, equipping LLMs with considerable capabilities for code-aided mathematical reasoning. However, continually training these models on augmented data derived from a few datasets such as GSM8K may impair their generalization abilities and restrict their effectiveness to a narrow range of question types. Conversely, the potential of improving such LLMs by leveraging large-scale, expert-written, diverse math question-answer pairs remains une…
-
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language ModelsL. Yu, Weisen Jiang, Han Shi et al. · 2023 · International Conference on Learning Representations · cité 790 fois · Pas encore résumé
Résumé scientifique
Large language models (LLMs) have pushed the limits of natural language understanding and exhibited excellent problem-solving ability. Despite the great success, most existing open-source LLMs (e.g., LLaMA-2) are still far away from satisfactory for solving mathematical problem due to the complex reasoning procedures. To bridge this gap, we propose MetaMath, a fine-tuned language model that specializes in mathematical reasoning. Specifically, we start by bootstrapping mathematical questions by rewriting the question from multiple perspectives without extra knowledge, which results in a new dataset called MetaMathQA. Then we fine-tune the LLaMA-2 models on MetaMathQA. Experimental results on …
-
DeepCritic: Deliberate Critique with Large Language ModelsWenkai Yang, Jingwen Chen, Yankai Lin et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 16 fois · Pas encore résumé
Résumé scientifique
As Large Language Models (LLMs) are rapidly evolving, providing accurate feedback and scalable oversight on their outputs becomes an urgent and critical problem. Leveraging LLMs as critique models to achieve automated supervision is a promising solution. In this work, we focus on studying and enhancing the math critique ability of LLMs. Current LLM critics provide critiques that are too shallow and superficial on each step, leading to low judgment accuracy and struggling to offer sufficient feedback for the LLM generator to correct mistakes. To tackle this issue, we propose a novel and effective two-stage framework to develop LLM critics that are capable of deliberately critiquing on each re…
-
Can LLMs Master Math? Investigating Large Language Models on Math Stack ExchangeAnkit Satpute, Noah Gießing, André Greiner-Petter et al. · 2024 · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · cité 57 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) have demonstrated exceptional capabilities in various natural language tasks, often achieving performances that surpass those of humans. Despite these advancements, the domain of mathematics presents a distinctive challenge, primarily due to its specialized structure and the precision it demands. In this work, we follow a two-step approach to investigating the proficiency of LLMs in answering mathematical questions. First, we employ the most effective LLMs, as identified by their performance on math question-answer benchmarks, to generate answers to 78 questions from the Math Stack Exchange (MSE). Second, a case analysis is conducted on the LLM that showed the hi…
-
FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language ModelsYan Liu, Ren-Ren Jin, Ling Shi et al. · 2024 · ACM Trans. Asian Low Resour. Lang. Inf. Process. · cité 9 fois · Pas encore résumé
Résumé scientifique
To thoroughly assess the mathematical reasoning abilities of Large Language Models (LLMs), we need to carefully curate evaluation datasets covering diverse mathematical concepts and mathematical problems at different difficulty levels. In pursuit of this objective, we propose FineMath in this article, a fine-grained mathematical evaluation benchmark dataset for assessing Chinese LLMs. FineMath is created to cover the major key mathematical concepts taught in elementary school math, which are further divided into 17 categories of math word problems, enabling in-depth analysis of mathematical reasoning abilities of LLMs. All the 17 categories of math word problems are manually annotated with t…
-
Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time SupervisionZhiheng Xi, Dingwen Yang, Jixuan Huang et al. · 2024 · Computation and Language · preprint, non relu par les pairs · cité 55 fois · Résumé IA disponible
Résumé scientifique
Training large language models (LLMs) to spend more time thinking and reflection before responding is crucial for effectively solving complex reasoning tasks in fields such as science, coding, and mathematics. However, the effectiveness of mechanisms like self-reflection and self-correction depends on the model's capacity to accurately assess its own performance, which can be limited by factors such as initial accuracy, question difficulty, and the lack of external feedback. In this paper, we delve into a two-player paradigm that separates the roles of reasoning and critique models, where the critique model provides step-level feedback to supervise the reasoning (actor) model during both tes…
-
WarriorMath: Enhancing the Mathematical Ability of Large Language Models with a Defect-aware FrameworkYue Chen, Ming-Hua He, Fangkai Yang et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 5 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) excel in solving mathematical problems, yet their performance is often limited by the availability of high-quality, diverse training data. Existing methods focus on augmenting datasets through rephrasing or difficulty progression but overlook the specific failure modes of LLMs. This results in synthetic questions that the model can already solve, providing minimal performance gains. To address this, we propose WarriorMath, a defect-aware framework for mathematical problem solving that integrates both targeted data synthesis and progressive training. In the synthesis stage, we employ multiple expert LLMs in a collaborative process to generate, critique, and refine…
Les travaux antérieurs sur lesquels il s’appuie
-
MAmmoTH: Building Math Generalist Models through Hybrid Instruction TuningXiang Yue, Xing-Wei Qu, Ge Zhang et al. · 2023 · International Conference on Learning Representations · cité 641 fois · Pas encore résumé
Résumé scientifique
We introduce MAmmoTH, a series of open-source large language models (LLMs) specifically tailored for general math problem-solving. The MAmmoTH models are trained on MathInstruct, our meticulously curated instruction tuning dataset. MathInstruct is compiled from 13 math datasets with intermediate rationales, six of which have rationales newly curated by us. It presents a unique hybrid of chain-of-thought (CoT) and program-of-thought (PoT) rationales, and also ensures extensive coverage of diverse fields in math. The hybrid of CoT and PoT not only unleashes the potential of tool use but also allows different thought processes for different math problems. As a result, the MAmmoTH series substan…
-
Black-Box Prompt Optimization: Aligning Large Language Models without Model TrainingJiale Cheng, Xiao Liu, Ke-Han Zheng et al. · 2023 · Annual Meeting of the Association for Computational Linguistics · cité 178 fois · Pas encore résumé
Résumé scientifique
Large language models (LLMs) have shown impressive success in various applications. However, these models are often not well aligned with human intents, which calls for additional treatments on them; that is, the alignment problem. To make LLMs better follow user instructions, existing alignment methods primarily focus on further training them. However, the extra training of LLMs is usually expensive in terms of GPU computing; even worse, some LLMs are not accessible for user-demanded training, such as GPTs. In this work, we take a different perspective -- Black-Box Prompt Optimization (BPO) -- to perform alignments. The idea is to optimize user prompts to suit LLMs' input understanding, so …
-
CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model GenerationPei Ke, Bo-Si Wen, Zhuo-Er Feng et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 61 fois · Pas encore résumé
-
How well do Large Language Models perform in Arithmetic tasks?Zheng Yuan, Hongyi Yuan, Chuan-Qi Tan et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 188 fois · Pas encore résumé
Résumé scientifique
Large language models have emerged abilities including chain-of-thought to answer math word problems step by step. Solving math word problems not only requires abilities to disassemble problems via chain-of-thought but also needs to calculate arithmetic expressions correctly for each step. To the best of our knowledge, there is no work to focus on evaluating the arithmetic ability of large language models. In this work, we propose an arithmetic dataset MATH 401 to test the latest large language models including GPT-4, ChatGPT, InstrctGPT, Galactica, and LLaMA with various arithmetic expressions and provide a detailed analysis of the ability of large language models. MATH 401 and evaluation c…
-
MathAttack: Attacking Large Language Models Towards Math Solving AbilityZihao Zhou, Qiufeng Wang, Mingyu Jin et al. · 2023 · AAAI Conference on Artificial Intelligence · cité 44 fois · Pas encore résumé
Résumé scientifique
With the boom of Large Language Models (LLMs), the research of solving Math Word Problem (MWP) has recently made great progress. However, there are few studies to examine the robustness of LLMs in math solving ability. Instead of attacking prompts in the use of LLMs, we propose a MathAttack model to attack MWP samples which are closer to the essence of robustness in solving math problems. Compared to traditional text adversarial attack, it is essential to preserve the mathematical logic of original MWPs during the attacking. To this end, we propose logical entity recognition to identify logical entries which are then frozen. Subsequently, the remaining text are attacked by adopting a word-le…
Les articles qui le citent
-
LEMMA: Learning from Errors for MatheMatical Advancement in LLMsZhuo-Shi Pan, Yu Li, Hong-Lin Lin et al. · 2025 · Annual Meeting of the Association for Computational Linguistics · cité 36 fois · Pas encore résumé
Résumé scientifique
Large language models (LLMs) have demonstrated remarkable reasoning capability in solving mathematical problems. However, existing approaches primarily focus on improving the quality of correct training data, e.g., distilling high-quality correct solutions from advanced models, neglecting the value contained in error data, potentially hindering the model's reflective ability. Though some studies attempt to leverage error data, they often involve complex mechanisms, such as Monte Carlo Tree Search (MCTS) to explore error nodes. In this work, we propose to enhance LLMs' reasoning ability by Learning from Errors for Mathematical Advancement (LEMMA). LEMMA constructs data consisting of an incorr…
-
A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & ChallengesYibo Yan, Jiamin Su, Jianxiang He et al. · 2024 · Annual Meeting of the Association for Computational Linguistics · cité 82 fois · Pas encore résumé
Résumé scientifique
Mathematical reasoning, a core aspect of human cognition, is vital across many domains, from educational problem-solving to scientific advancements. As artificial general intelligence (AGI) progresses, integrating large language models (LLMs) with mathematical reasoning tasks is becoming increasingly significant. This survey provides the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models (MLLMs). We review over 200 studies published since 2021, and examine the state-of-the-art developments in Math-LLMs, with a focus on multimodal settings. We categorize the field into three dimensions: benchmarks, methodologies, and challenges. In particular…
-
Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models - The Story Goes OnLiang Zeng, Liangjun Zhong, Liang Zhao et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 25 fois · Pas encore résumé
Résumé scientifique
In this paper, we investigate the underlying factors that potentially enhance the mathematical reasoning capabilities of large language models (LLMs). We argue that the data scaling law for math reasoning capabilities in modern LLMs is far from being saturated, highlighting how the model's quality improves with increases in data quantity. To support this claim, we introduce the Skywork-Math model series, supervised fine-tuned (SFT) on common 7B LLMs using our proposed 2.5M-instance Skywork-MathQA dataset. Skywork-Math 7B has achieved impressive accuracies of 51.2% on the competition-level MATH benchmark and 83.9% on the GSM8K benchmark using only SFT data, outperforming an early version of G…
-
MathScape: Evaluating MLLMs in multimodal Math Scenarios through a Hierarchical BenchmarkMinxuan Zhou, Hao Liang, Tian-Peng Li et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 20 fois · Pas encore résumé
-
CriticEval: Evaluating Large Language Model as CriticTian Lan, Wen-Wei Zhang, Chen Xu et al. · 2024 · cité 12 fois · Pas encore résumé
Résumé scientifique
Critique ability, i.e., the capability of Large Language Models (LLMs) to identify and rectify flaws in responses, is crucial for their applications in self-improvement and scalable oversight. While numerous studies have been proposed to evaluate critique ability of LLMs, their comprehensiveness and reliability are still limited. To overcome this problem, we introduce CriticEval, a novel benchmark designed to comprehensively and reliably evaluate critique ability of LLMs. Specifically, to ensure the comprehensiveness, CriticEval evaluates critique ability from four dimensions across nine diverse task scenarios. It evaluates both scalar-valued and textual critiques, targeting responses of var…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 2 novembre 2026.