# Learning from Dialogue after Deployment: Feed Yourself, Chatbot!

**Authors:** Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, Jason Weston  
**arXiv:** [1901.05415](https://arxiv.org/abs/1901.05415)  
**Date:** 2019-01-16  
**Summary page:** https://www.summarizepaper.com/en/paper/1901.05415v1/

## Key points

- Issue of untapped training potential in dialogue agents
- Proposal of a self-feeding chatbot to extract new training examples from conversations
- Self-feeding chatbot estimates user satisfaction with responses
- User's responses used as new training examples for the chatbot to imitate when conversation is going well
- Chatbot asks for feedback to improve dialogue abilities when it believes it has made a mistake
- Learning to predict feedback enhances chatbot's performance
- Evaluation on PersonaChat chit-chat dataset with over 131k training examples
- Learning from dialogue with self-feeding chatbot improves performance regardless of traditional supervision provided
- Introduction of a novel method for dialogue agents to continuously learn and improve conversational abilities by leveraging real-world conversations.

## Summary

This paper by Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston addresses the issue of untapped training potential in dialogue agents. The majority of conversations that a dialogue agent encounters happen after it has already been trained and deployed. To address this, the authors propose a self-feeding chatbot that can extract new training examples from the conversations it engages in. The self-feeding chatbot not only participates in conversations but also estimates user satisfaction with its responses. When the conversation is going well, the user's responses are used as new training examples for the chatbot to imitate. On the other hand, when the chatbot believes it has made a mistake, it asks for feedback to improve its dialogue abilities further. By learning to predict the feedback that will be given, the chatbot enhances its performance. The authors evaluate their approach on the PersonaChat chit-chat dataset which contains over 131k training examples. They find that learning from dialogue with a self-feeding chatbot significantly improves performance regardless of the amount of traditional supervision provided. Overall, this work introduces a novel method for dialogue agents to continuously learn and improve their conversational abilities by leveraging real-world conversations they participate in.