# Sparks of Artificial General Intelligence: Early experiments with GPT-4

**Authors:** Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang  
**arXiv:** [2303.12712](https://arxiv.org/abs/2303.12712)  
**Date:** 2023-03-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.12712v2/

## Key points

- AI researchers have developed large language models (LLMs) with remarkable capabilities
- OpenAI's latest model, GPT-4, was trained using an unprecedented scale of compute and data
- Early version of GPT-4 is part of a new cohort of LLMs that exhibit more general intelligence than previous AI models
- GPT-4 can solve novel and difficult tasks spanning various domains without needing any special prompting
- GPT-4's performance is strikingly close to human-level performance and often surpasses prior models such as ChatGPT
- Challenges ahead for advancing towards deeper and more comprehensive versions of artificial general intelligence (AGI)
- Shortcomings in current metrics used to evaluate LLMs' performance as they fail to capture semantic similarities within statements but rely primarily on word or sentence-level similarity metrics which capture syntax.
- Examples where GPT-4 lacks planning abilities in text generation under constraints.

## Summary

In recent years, artificial intelligence (AI) researchers have been developing large language models (LLMs) that exhibit remarkable capabilities across various domains and tasks. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, the authors report on their investigation of an early version of GPT-4 when it was still in active development by OpenAI. They contend that this early version of GPT-4 is part of a new cohort of LLMs that exhibit more general intelligence than previous AI models. The authors demonstrate that beyond its mastery of language, GPT-4 can solve novel and difficult tasks spanning mathematics, coding, vision, medicine, law, psychology and more without needing any special prompting. Moreover, in all these tasks, GPT-4's performance is strikingly close to human-level performance and often vastly surpasses prior models such as ChatGPT. The authors put special emphasis on discovering the limitations of GPT-4 and discuss the challenges ahead for advancing towards deeper and more comprehensive versions of artificial general intelligence (AGI), including the possible need for pursuing a new paradigm that moves beyond next-word prediction. They conclude with reflections on societal influences of the recent technological leap and future research directions. The paper also highlights some shortcomings in current metrics used to evaluate LLMs' performance as they fail to capture semantic similarities within statements but rely primarily on word or sentence-level similarity metrics which capture syntax. Additionally, the paper discusses examples where GPT-4 lacks planning abilities in text generation under constraints. Overall, given the breadth and depth of GPT-4's capabilities demonstrated by the authors' investigation, it could reasonably be viewed as an early (yet still incomplete) version of an AGI system. However, there are still significant challenges ahead for advancing towards deeper and more comprehensive versions of AGI.