# TOFU: A Task of Fictitious Unlearning for LLMs

**Authors:** Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, J. Zico Kolter  
**arXiv:** [2401.06121](https://arxiv.org/abs/2401.06121)  
**Date:** 2024-01-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.06121v1/

## Key points

- Large language models (LLMs) trained on web data can memorize and reproduce sensitive or private information, raising legal and ethical concerns.
- The concept of unlearning is proposed as a solution to protect private data after training LLMs.
- Uncertainty exists regarding the effectiveness of existing unlearning methods in making models behave as if they were never trained on forgotten data.
- TOFU (Task of Fictitious Unlearning) is introduced as a benchmark to evaluate unlearning efficacy.
- TOFU includes a dataset of 200 synthetic author profiles with a forget set that serves as the target for unlearning.
- Baseline results from existing unlearning algorithms show ineffective unlearning.
- Continued efforts are needed to develop approaches for effective unlearning and address legal and ethical concerns associated with LLMs trained on web data.

## Summary

In their paper titled "TOFU: A Task of Fictitious Unlearning for LLMs," authors Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter discuss the concerns surrounding large language models (LLMs) trained on vast amounts of web data. These models have the ability to memorize and reproduce sensitive or private information, which raises both legal and ethical issues. To address this problem, the authors propose the concept of unlearning, which involves tuning LLMs to forget specific information present in their training data. Unlearning provides a way to protect private data after training the models. However, there is uncertainty regarding the extent to which existing unlearning methods can effectively result in models that behave as if they were never trained on the forgotten data. To deepen our understanding of unlearning and evaluate its efficacy, the authors introduce TOFU as a benchmark. TOFU stands for "Task of Fictitious Unlearning" and includes a dataset consisting of 200 diverse synthetic author profiles. Each profile comprises 20 question-answer pairs. Within this dataset, there is a subset called the forget set that serves as the target for unlearning. The authors compile a suite of metrics that work together to provide a comprehensive assessment of unlearning efficacy. They also present baseline results from existing unlearning algorithms but highlight that none of these baselines demonstrate effective unlearning. The findings from this study emphasize the need for continued efforts in developing approaches for unlearning that truly tune LLMs to behave as if they were never trained on the forgotten data at all. By providing this refined benchmark and highlighting current limitations, TOFU aims to contribute towards advancements in protecting sensitive information and addressing legal and ethical concerns associated with large language models trained on web data.