# Recipes for Safety in Open-domain Chatbots

**Authors:** Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan  
**arXiv:** [2010.07079](https://arxiv.org/abs/2010.07079)  
**Date:** 2020-10-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2010.07079v1/

## Key points

- Authors address offensive and biased behavior in open-domain chatbots
- Propose human-and-model-in-the-loop approach for training and evaluating safer models
- Introduce method to incorporate safety considerations directly into generative models
- Conduct experiments comparing their methods with existing models
- Results show their approaches are safer and maintain usability metrics
- Discuss limitations and failure cases of their models
- Provide insights for future research and development in the field

## Summary

In their paper titled "Recipes for Safety in Open-domain Chatbots," authors Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan address the issue of offensive and biased behavior exhibited by models trained on large unlabeled corpora of human interactions. They propose various methods to mitigate these problems within open-domain generative dialogue models. The authors introduce a novel framework called the human-and-model-in-the-loop approach, which is used for both training safer models and evaluating them. This framework incorporates human feedback during the training process to ensure that the models learn appropriate behaviors and avoid offensive or toxic language. Additionally, they present a new method to incorporate safety considerations directly into generative models without relying on external classifiers at deployment time. To validate the effectiveness of their techniques, the authors conduct experiments comparing their methods with existing models. The results demonstrate that their approaches are not only safer than previous models according to automatic and human evaluations but also maintain usability metrics such as engagingness relative to state-of-the-art models. Furthermore, the paper discusses potential limitations of their work by analyzing failure cases of their models. By identifying these shortcomings, the authors provide insights into areas where further improvements can be made in order to enhance model safety. Overall, this study presents valuable contributions towards addressing issues related to offensive behavior and biases in open domain chatbots. The proposed human-and-model-in-the loop framework and distillation method offer effective strategies for training safer dialogue models while maintaining usability metrics. The analysis of failure cases highlights potential areas for future research and development in this field.