# Facilitating NSFW Text Detection in Open-Domain Dialogue Systems via Knowledge Distillation

**Authors:** Huachuan Qiu, Shuai Zhang, Hongliang He, Anqi Li, Zhenzhong Lan  
**arXiv:** [2309.09749](https://arxiv.org/abs/2309.09749)  
**Date:** 2023-09-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.09749v1/

## Key points

- The paper addresses the issue of NSFW content in open-domain dialogue systems and its negative impact on users.
- Detecting NSFW language, particularly sexually explicit content, within dialogue context has been overlooked.
- The authors introduce CensorChat, a dialogue monitoring dataset for NSFW dialogue detection.
- Knowledge distillation techniques involving GPT-4 and ChatGPT are used to construct cost-effective NSFW content detectors.
- Real-life human-machine interaction data is collected and broken down into single utterances and dialogues for annotation.
- ChatGPT and GPT-4 are used as annotators to construct rationale validation and test sets.
- A self-criticism strategy is employed to resolve discrepancies in labeling.
- A BERT model is fine-tuned as a text classifier using pseudo-labeled data; its performance is evaluated.
- User safety and well-being in digital conversations should be prioritized while respecting freedom of expression within AI systems.
- Advancing NSFW content detection capabilities aligns with evolving user protection needs in AI-driven dialogues.

## Summary

The paper titled "Facilitating NSFW Text Detection in Open-Domain Dialogue Systems via Knowledge Distillation" addresses the issue of Not Safe for Work (NSFW) content in open-domain dialogue systems and its potential negative impact on users. The research highlights that while detecting NSFW language, particularly sexually explicit content, within a dialogue context is crucial, it has been significantly overlooked. To tackle this problem, the authors introduce CensorChat—a dialogue monitoring dataset specifically designed for NSFW dialogue detection. The dataset leverages knowledge distillation techniques involving GPT-4 and ChatGPT to construct cost-effective NSFW content detectors. The process involves collecting real-life human-machine interaction data and breaking it down into single utterances and single-turn dialogues with the chatbot delivering the final utterance. In order to annotate unlabeled data, ChatGPT is utilized as a training set; rationale validation and test sets are constructed using both ChatGPT and GPT-4 as annotators. To resolve discrepancies in labeling, a self-criticism strategy is employed. Additionally, a BERT model is fine-tuned as a text classifier using pseudo-labeled data; its performance is evaluated. The study emphasizes the importance of prioritizing user safety and well-being in digital conversations while respecting freedom of expression within AI systems. By advancing NSFW content detection capabilities, the proposed approach aligns with evolving user protection needs in AI-driven dialogues. Overall, this research contributes to addressing the lag in detecting NSFW language within dialogue contexts by introducing CensorChat—a valuable dataset—and employing knowledge distillation techniques. It underscores the significance of developing AI systems that prioritize user safety while facilitating meaningful conversations online.