# DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

**Authors:** Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn  
**arXiv:** [2301.11305](https://arxiv.org/abs/2301.11305)  
**Date:** 2023-01-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.11305v1/

## Key points

- Large language models (LLMs) like GPT-3 are capable of generating highly fluent and factually accurate text.
- Detecting machine-written content has become increasingly important, especially in academic settings where students may use LLMs to complete assignments.
- DetectGPT is a new approach that leverages negative curvature regions in the log probability function of LLMs to detect machine-generated text.
- DetectGPT compares the log probability of a passage with perturbations from a generic pre-trained model like T5 to determine if it was likely generated by a specific LLM such as GPT-3.
- This approach does not require training a separate classifier or collecting datasets of real or generated passages.
- Machine-generated text extends beyond academia into journalism and other contexts, raising concerns about factual accuracy and potential misinformation.
- Automated detection methods like DetectGPT provide teachers and news-readers with more confidence in determining the origin of text they consume.
- DetectGPT represents a significant advancement in zero-shot machine-generated text detection, outperforming existing methods in identifying fake news articles generated by large-scale LLMs like GPT-NeoX.
- Tools like DetectGPT address concerns related to student assessment, misinformation, and pave the way for more reliable interactions with machine-generated content across various domains.

## Summary

In a world where large language models (LLMs) like GPT-3 are capable of generating highly fluent and factually accurate text, the need for systems to detect machine-written content has become increasingly important. This is particularly evident in scenarios where students may use LLMs to complete assignments, making it challenging for instructors to assess student learning accurately. In response to this challenge, a new approach called DetectGPT has been developed. <br>
DetectGPT leverages the observation that text generated by LLMs tends to occupy negative curvature regions of the model's log probability function. By comparing the log probability of a candidate passage with minor perturbations generated by a generic pre-trained model like T5, DetectGPT can determine if the passage is likely generated from a specific LLM such as GPT-3. <br>
Importantly, this approach does not require training a separate classifier or collecting datasets of real or generated passages. The implications of machine-generated text extend beyond academic settings, with AI-written content being used in journalism and other contexts. However, concerns arise regarding factual accuracy and the potential for misleading information in articles produced by LLMs with limited human review. Automated detection methods like DetectGPT offer a solution to these challenges by providing teachers and news-readers with more confidence in determining the origin of the text they consume. Overall, DetectGPT represents a significant advancement in zero-shot machine-generated text detection, outperforming existing methods in identifying fake news articles generated by large-scale LLMs like GPT-NeoX.<br>
The development of such tools not only addresses immediate concerns related to student assessment and misinformation but also paves the way for more reliable and trustworthy interactions with machine-generated content in various domains.