# Extracting Training Data from Large Language Models

**Authors:** Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel  
**arXiv:** [2012.07805](https://arxiv.org/abs/2012.07805)  
**Date:** 2020-12-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2012.07805v2/

## Key points

- Large language models with billions of parameters are increasingly being developed and deployed in artificial intelligence.
- Concerns have been raised about the security implications associated with training these models on vast amounts of data, including private datasets.
- A study by Nicholas Carlini and team revealed a vulnerability where malicious entities can extract specific examples from a model's training data through querying, as demonstrated on GPT-2.
- The researchers successfully extracted sensitive information like personally identifiable details, IRC conversations, code snippets, and 128-bit UUIDs from GPT-2's training data.
- Larger language models are more susceptible to such extraction attacks compared to smaller models according to the study's findings.
- Developers and organizations using large language models should implement robust safeguards and protocols during training processes to mitigate risks of unauthorized access or leakage of sensitive information.

## Summary

In the realm of artificial intelligence, the development and deployment of large language models have become increasingly prevalent. These models, often containing billions of parameters, are typically trained on vast amounts of data, including private datasets. However, a concerning revelation has emerged regarding the security implications associated with such practices. A recent study conducted by Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea and Colin Raffel sheds light on a potential vulnerability inherent in these large language models. The researchers demonstrate that malicious entities can exploit a training data extraction attack to retrieve specific examples from the model's training data simply by querying it. The focus of their investigation was GPT-2 - a prominent language model trained on snippets gathered from the public Internet. Through their attack methodology,the team was able to extract numerous verbatim text sequences from GPT-2's training data. These extracted examples encompassed various forms of sensitive information such as personally identifiable details (names,p hone numbers,email addresses), IRC conversations,c ode snippets,and 128-bit UUIDs. Remarkably,this extraction was successful even when each sequence appeared only once in the training data. To gain deeper insights into the factors influencing the efficacy of their extraction attack comprehensive evaluations were carried out by the researchers.Alarmingly,their findings indicated that larger language models are more susceptible to such attacks compared to smaller counterparts.As a result of their study outcomes and observations made during experimentation with GPT-2's vulnerabilities to data extraction attacks,it is imperative for developers and organizations utilizing large language models to implement robust safeguards and protocols during training processes to mitigate potential risks associated with unauthorized access or leakage of sensitive information. This research serves as a crucial reminder of the importance of prioritizing security measures in AI development and deployment practices.