# Enhance Reasoning for Large Language Models in the Game Werewolf

**Authors:** Shuang Wu, Liwen Zhu, Tao Yang, Shiwei Xu, Qiang Fu, Yang Wei, Haobo Fu  
**arXiv:** [2402.02330](https://arxiv.org/abs/2402.02330)  
**Date:** 2024-02-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.02330v1/

## Key points

- A novel framework proposed to enhance reasoning capabilities of Large Language Models (LLMs) by integrating them with an external Thinker module
- Thinker module leverages knowledge from databases and optimization techniques for complex logical analysis and domain-specific tasks
- Hierarchy established where LLMs focus on System-1 tasks, while Thinker specializes in cognitive System-2 reasoning
- Framework demonstrated effectiveness in a 9-player Werewolf game scenario requiring dual-system reasoning
- Communication protocol introduced to facilitate interaction between LLMs and the Thinker, trained using data from over 18,800 human game sessions and reinforcement learning techniques
- Experimental results showcase proficiency in deductive reasoning, speech generation, and online game evaluation
- 6B LLM integrated with the Thinker surpasses GPT4 after fine-tuning
- Largest dataset provided for social deduction games to date through authentic game sessions and speech data collection methodology
- Framework comprises three key components: The Listener for natural language understanding, the Thinker for deep logical analysis and decision-making, and the Presenter for generating coherent language output based on strategic instructions from the Thinker
- Data preparation involved collecting game data from Werewolf sessions enriched with domain-specific corpus sources and utilizing advanced models like Paraformer for Automatic Speech Recognition (ASR)
- Study aims to develop more generalized and flexible methods by incorporating data-driven approaches
- Evaluation involves testing AI performance in games involving AI vs AI or one human player against multiple AIs; challenges arise in evaluating AI performance in settings with human players due to interactive nature and varied speech strategies

## Summary

A novel framework is proposed to enhance the reasoning capabilities of Large Language Models (LLMs) by integrating them with an external Thinker module. Unlike traditional methods that rely on prompt engineering, the Thinker module leverages knowledge from databases and employs optimization techniques to handle complex logical analysis and domain-specific tasks. The framework establishes a hierarchy where LLMs focus on intuitive System-1 tasks like natural language processing, while the Thinker specializes in cognitive System-2 reasoning. The effectiveness of this framework is demonstrated through its application in a 9-player Werewolf game scenario that requires dual-system reasoning. A communication protocol is introduced to facilitate interaction between LLMs and the Thinker, with the latter being trained using data from over 18,800 human game sessions and reinforcement learning techniques. Experimental results showcase the framework's proficiency in deductive reasoning, speech generation, and online game evaluation. Furthermore, a 6B LLM integrated with the Thinker surpasses GPT4 after fine-tuning. This study also contributes to the field by providing the largest dataset for social deduction games to date. The methodology involves collecting authentic game sessions and speech data to align closely with real-world scenarios and human interaction patterns. The framework comprises three key components: The Listener for natural language understanding, the Thinker for deep logical analysis and decision-making, and the Presenter for generating coherent language output based on strategic instructions from the Thinker. Data preparation involved collecting game data from Werewolf sessions hosted on an online platform, enriching it with domain-specific corpus sources, and utilizing advanced models like Paraformer for Automatic Speech Recognition (ASR). Overall, this study aims to develop more generalized and flexible methods by incorporating data-driven approaches. Evaluation involves testing AI performance in games involving AI vs AI or one human player against multiple AIs. Challenges arise in evaluating AI performance in settings with human players due to their interactive nature and varied speech strategies.