# ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection

**Authors:** Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, Ece Kamar  
**arXiv:** [2203.09509](https://arxiv.org/abs/2203.09509)  
**Date:** 2022-03-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.09509v1/

## Key points

- Toxic language detection systems struggle to identify toxic text targeting minority groups due to spurious correlations.
- ToxiGen is a large-scale machine-generated dataset consisting of 274k toxic and benign statements about 13 minority groups.
- ToxiGen is generated using a demonstration-based prompting framework and an adversarial classifier-in-the-loop decoding method.
- Human evaluation shows that annotators cannot distinguish between machine-generated text and human-written language.
- 94.5% of toxic examples in ToxiGen are labeled as hate speech by human annotators.
- Finetuning a toxicity classifier on ToxiGen data significantly improves its performance on human-written data.
- Experiments with three publicly available datasets demonstrate the improvement achieved through finetuning on ToxiGen data.
- ToxiGen is presented as a valuable resource for combating machine-generated toxicity and advancing hate speech detection systems.
- The study highlights the potential benefits of incorporating machine-generated data into training models for accurate identification of toxic language.

## Summary

Toxic language detection systems often struggle to accurately identify toxic text that targets minority groups due to spurious correlations. To address this challenge, the authors introduce ToxiGen - a large-scale machine-generated dataset consisting of 274k toxic and benign statements about 13 minority groups. This dataset is generated using a demonstration-based prompting framework and an adversarial classifier-in-the-loop decoding method. A human evaluation is conducted on a challenging subset of the dataset which reveals that annotators are unable to distinguish between machine-generated text and human-written language. Additionally, 94.5% of toxic examples are labeled as hate speech by human annotators. Finetuning a toxicity classifier on ToxiGen data enhances its performance significantly when applied to human-written data with experiments using three publicly available datasets demonstrating this improvement. This study presents ToxiGen as a valuable resource for combating machine-generated toxicity and advancing hate speech detection systems. The findings highlight the potential benefits of incorporating machine-generated data into training models to improve their performance in identifying toxic language accurately.