Exécution de notre outil de synthèse sur un nouvel article

Vérifiez l’article ci-dessous, puis lancez sa synthèse.


Empowering Large Language Model Agent through Step-Level Self-Critique and Self-Training

Yuanzhao Zhai (National University of Defense Technology), Huanxi Liu (National University of Defense Technology), Zhuo Zhang (Harbin Institute of Technology, Peng Cheng Laboratory), Tong Lin (National University of Defense Technology), Kele Xu (National University of Defense Technology), Cheng Yang (National University of Defense Technology), Dawei Feng (National University of Defense Technology), Bo Ding (National University of Defense Technology), Huaimin Wang (National University of Defense Technology)

International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)
DOI: 10.1145/3726302.3729965 (Computer Science)
Licence : non indiquée (tous droits réservés)

Résumé : Large Language Model (LLM) agents frequently produce sub-optimal actions when tackling complex, multi-step decision-making tasks. Employing self-critique to identify flaws and suggest enhancements is an effective strategy for refining actions. Although trajectory-level critique is commonly employed, it often fails to identify flawed steps accurately. In this paper, we introduce SLSC-MCTS, a method that integrates Monte Carlo Tree Search with Step-Level Self-Critique to enhance LLM agents during both testing and self-training phases. During decision tree expansion with SLSC-MCTS, the LLM agent initially generates an action, receives environmental feedback, and subsequently generates further actions via self-critique and refinement. Through multiple episodes of SLSC-MCTS, LLM agents can effectively utilize step-level critiques while disregarding ineffective ones based on node values, thereby incorporating the critiques more robustly. Additionally, our method further empowers LLM agents in a self-training manner, collecting training data from the constructed decision tree to iteratively fine-tune the LLM agents. The self-training data gathered via SLSC-MCTS is diverse and high-quality, which further enhances the reasoning, critiquing, and refining abilities of LLM agents. Experimental results demonstrate that SLSC-MCTS significantly improves LLM agents during testing, surpassing state-of-the-art baselines and achieving shorter task completion trajectories across information retrieval benchmarks such as WebShop and HotPotQA. After three iterations of self-training, LLM agents established by Llama-3.1-8B-Instruct show substantial improvement, even surpassing human experts in WebShop.

Publié le 13 Jul. 2025