# GPT-4 is judged more human than humans in displaced and inverted Turing tests

**Authors:** Ishika Rathi, Sydney Taylor, Benjamin K. Bergen, Cameron R. Jones  
**arXiv:** [2407.08853](https://arxiv.org/abs/2407.08853)  
**Date:** 2024-07-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2407.08853v1/

## Key points

- Study focuses on challenges of detecting everyday AI in informal online conversations
- Modified versions of Turing test conducted to measure ability to discriminate between human and AI interactions
- Judges included GPT-3.5, GPT-4, and displaced humans, all showed below chance accuracy
- Best-performing GPT-4 witness often judged as human more than actual humans
- AI system may be perceived as human more often than actual person in online conversations
- Transcript length had counter-intuitive effect on accuracy, shorter transcripts potentially more helpful due to biases in length determination
- Differences in how human adjudicators completed transcripts compared to LLM adjudicators highlighted factors influencing judgment accuracy

## Summary

The study delves into the challenges of detecting everyday AI in informal online conversations. Distinguishing between human and AI interactions is crucial in this context. To measure people's and large language models' (LLMs) ability to discriminate between human and AI agents, modified versions of the Turing test were conducted. These included inverted and displaced tests. The judges tasked with determining whether an agent was human or AI were GPT-3.5, GPT-4, and displaced humans. Surprisingly, all three groups exhibited lower accuracy compared to interactive interrogators, with overall below chance accuracy. Strikingly, they tended to judge the best-performing GPT-4 witness as human more often than actual human witnesses. This highlights the difficulty for both humans and current LLMs in distinguishing between human and AI interactions without active interrogation. Further analysis revealed that the best-performing GPT-4 witness had a higher pass rate than human witnesses in both inverted and displaced tests. This suggests that in online conversations between humans and AI models, the AI system may be more likely to be perceived as human than an actual person. Additionally, a counter-intuitive effect of transcript length on accuracy was found - shorter transcripts may contain information more helpful to adjudicators due to potential biases in how transcript length was determined. Moreover, differences in how human adjudicators completed transcripts in series compared to LLM adjudicators who assessed each transcript separately highlighted potential factors influencing judgment accuracy. Overall, this study emphasizes the need for improved tools for detecting AI in conversations given the challenges faced by both humans and current LLMs in accurately discerning between human and AI interactions without active interrogation.