# GPT-4 Technical Report

**Authors:**  OpenAI  
**arXiv:** [2303.08774](https://arxiv.org/abs/2303.08774)  
**Date:** 2023-03-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.08774v3/

## Key points

- OpenAI has released a technical report on the development of GPT-4, a large-scale, multimodal model that can accept both image and text inputs and produce text outputs.
- GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers.
- GPT-4 is based on the Transformer architecture and is pre-trained to predict the next token in a document.
- The post-training alignment process leads to improved performance on measures of factuality and adherence to desired behavior.
- OpenAI was able to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute used for training GPT-4 itself.
- The development of GPT-4 represents an important step forward in natural language processing research. Its ability to handle multimodal inputs makes it well-suited for a wide range of applications, from chatbots to automated content creation.
- There are potential risks associated with its use such as bias or unintended consequences. Continued research into ethical considerations surrounding AI development will be crucial moving forward.

## Summary

OpenAI has released a technical report on the development of GPT-4, a large-scale, multimodal model that can accept both image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is based on the Transformer architecture and is pre-trained to predict the next token in a document. The post-training alignment process leads to improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. One notable aspect of this project is that OpenAI was able to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute used for training GPT-4 itself. This suggests that OpenAI has made significant progress in developing infrastructure and optimization methods that allow them to scale up their models efficiently. Overall, the development of GPT-4 represents an important step forward in natural language processing research. Its ability to handle multimodal inputs makes it well-suited for a wide range of applications, from chatbots to automated content creation. However, as with any AI system, there are also potential risks associated with its use such as bias or unintended consequences. As such, continued research into ethical considerations surrounding AI development will be crucial moving forward.