# Does GPT-4 Pass the Turing Test?

**Authors:** Cameron Jones, Benjamin Bergen  
**arXiv:** [2310.20216](https://arxiv.org/abs/2310.20216)  
**Date:** 2023-10-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.20216v1/

## Key points

- GPT-4 passed in 41% of games in a public online Turing Test
- Surpassed baselines set by ELIZA (27%) and GPT-3.5 (14%)
- Fell short of chance and baseline set by human participants (63%)
- Decisions made by participants based on linguistic style (35%) and socio-emotional traits (27%)
- Participants' demographics did not predict ability to detect AI deception
- Turing Test remains relevant for assessing naturalistic communication and deception
- AI models masquerading as humans could have significant societal consequences
- Importance of analyzing different strategies and criteria for judging humanlikeness
- Insights into reasons behind participants' decisions when identifying an AI
- Concerns about potential widespread implications if AI systems convincingly mimic human behavior

## Summary

In a public online Turing Test, GPT-4 was evaluated and the best-performing GPT-4 prompt passed in 41% of games, surpassing the baselines set by ELIZA (27%) and GPT-3.5 (14%). However, it fell short of chance and the baseline set by human participants (63%). The decisions made by participants were primarily based on linguistic style (35%) and socio-emotional traits (27%), indicating that intelligence alone is not sufficient to pass the Turing Test. Surprisingly, participants' demographics did not predict their ability to detect AI deception. This suggests that even individuals who possess deep understanding of these systems and frequent interactions with them can be deceived. The study argues that despite its known limitations as a test of intelligence, the Turing Test remains relevant for assessing naturalistic communication and deception. The ability of AI models to masquerade as humans could have significant societal consequences, making it crucial to analyze different strategies and criteria for judging humanlikeness. The study also provides insights into various reasons behind participants' decisions when identifying an AI as well as their success rates in different games. Overall, this research highlights the importance of evaluating AI models' performance in terms of naturalistic communication skills and raises concerns about potential widespread implications if AI systems are able to convincingly mimic human behavior.