# Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions

**Authors:** Chen Feng Tsai, Xiaochen Zhou, Sierra S. Liu, Jing Li, Mo Yu, Hongyuan Mei  
**arXiv:** [2304.02868](https://arxiv.org/abs/2304.02868)  
**Date:** 2023-04-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.02868v1/

## Key points

- Large language models (LLMs) like ChatGPT and GPT-4 are highly skilled at communicating with humans.
- Researchers from multiple institutions conducted a technical report investigating LLMs' abilities to play text games.
- ChatGPT performs competitively compared to other systems, but still exhibits a low level of intelligence in certain areas.
- ChatGPT cannot construct a world model by playing the game or reading the manual, may fail to leverage existing knowledge, and cannot infer the goal of each step as the game progresses.
- There is currently a lack of certain fundamental properties of intelligence in ChatGPT, although some may emerge in future LLMs as they grow larger.
- Establishing a benchmark to evaluate these properties and tracking their potential advancement is desirable, and researchers are working on this.
- These results raise new research questions about LLMs' capabilities for playing text games at the intersection of artificial intelligence, machine learning, and natural language processing.

## Summary

The rise of large language models (LLMs) such as ChatGPT and GPT-4 has demonstrated their remarkable abilities to communicate with human users. A technical report by researchers from the University of Chicago, Syracuse University, New Jersey Institute of Technology, WeChat AI, and Toyota Technological Institute at Chicago investigates their capacities for playing text games. The report reveals that ChatGPT performs competitively compared to all existing systems but still exhibits a low level of intelligence. Specifically, it cannot construct the world model by playing the game or even reading the game manual; it may fail to leverage its existing world knowledge; and it cannot infer the goal of each step as the game progresses. While some properties may emerge from future LLMs as they grow larger, there is currently a lack of certain fundamental properties of intelligence in ChatGPT. The researchers believe that establishing a benchmark to evaluate such properties and tracking their potential advancement is desirable and are diligently working on this. These results open up new research questions at the intersection of artificial intelligence, machine learning, and natural language processing regarding LLMs' capabilities for playing text games.