# Deep Learning Based Chatbot Models

**Authors:** Richard Csaky  
**arXiv:** [1908.08835](https://arxiv.org/abs/1908.08835)  
**Date:** 2019-08-23  
**Summary page:** https://www.summarizepaper.com/en/paper/1908.08835v1/

## Key points

- The paper explores the field of conversational agents, specifically chatbots
- Emphasizes the importance of modeling conversation in natural language processing and AI
- Conducts an extensive survey of over 70 publications related to chatbots
- Argues that current state-of-the-art architectures fail to consider sufficient prior information when generating responses
- Highlights the influence of external sources such as persona or mood on conversation context
- Proposes ideas on addressing the problem of insufficient prior information
- Adapts the Transformer model for chatbot applications and conducts experiments using conversations from Cornell Movie-Dialog Corpus
- Augments the model with additional features like mood or persona alongside raw conversation data
- Analyzes the performance of the vanilla model compared to previous chatbot models and examines the impact of incorporating additional features on response quality
- Aims to improve chatbot models' ability to generate relevant and high-quality responses in various conversational contexts

## Summary

In the paper titled "Deep Learning Based Chatbot Models" by Richard Csaky, the author explores the field of conversational agents, specifically chatbots, which are software programs capable of communicating with humans using natural language. The paper emphasizes the importance of modeling conversation in natural language processing and artificial intelligence. Csaky conducts an extensive survey of recent literature, analyzing over 70 publications related to chatbots published within the last three years. Through this analysis, he argues that the nature of general conversation demands approaches that differ from current state-of-the-art architectures. He provides examples from existing literature to demonstrate how current chatbot models fail to consider sufficient prior information when generating responses, ultimately affecting the quality of conversation. The author highlights that these priors can include external sources of information such as persona or mood which heavily influence the context in which conversations occur. In addition to identifying this problem, Csaky proposes several ideas on how it can be addressed. The subsequent section focuses on adapting the Transformer model, a state-of-the-art neural machine translation architecture, to suit chatbot applications. The author presents experiments using a vanilla model with conversations extracted from the Cornell Movie-Dialog Corpus. Furthermore, Csaky augments the model by incorporating additional features like mood or persona alongside raw conversation data. Finally, a detailed analysis is conducted to evaluate how well the vanilla model performs on conversational data compared to previous chatbot models and examine the impact of incorporating additional features on response quality. This comprehensive study aims to contribute towards improving chatbot models and enhancing their ability to generate relevant and high-quality responses in various conversational contexts. Overall, Csaky's research provides valuable insights into deep learning-based chatbot models and offers potential solutions for addressing their limitations in capturing important contextual information during conversations.