# Building a Production Model for Retrieval-Based Chatbots

**Authors:** Kyle Swanson, Lili Yu, Christopher Fox, Jeremy Wohlwend, Tao Lei  
**arXiv:** [1906.03209](https://arxiv.org/abs/1906.03209)  
**Date:** 2019-06-07  
**Summary page:** https://www.summarizepaper.com/en/paper/1906.03209v1/

## Key points

- Authors address the task of response suggestion in human-computer conversation systems
- Little attention has been given to practicality in a production setting
- Dual encoder architecture proposed for rapid inference and scalability
- Two methods for generating whitelists introduced and compared
- Comprehensive analysis of model and whitelists conducted
- Experimental results demonstrate production-quality performance through offline metrics and human evaluation
- Study provides key lessons about conversation modeling in practice
- Emphasizes importance of considering practicality when developing conversation systems
- Contributes to building more efficient and reliable retrieval-based chatbots for real-world applications

## Summary

In the paper titled "Building a Production Model for Retrieval-Based Chatbots," authors Kyle Swanson, Lili Yu, Christopher Fox, Jeremy Wohlwend and Tao Lei address the task of response suggestion in human-computer conversation systems. While recent approaches to conversation modeling have shown impressive results with new model architectures, little attention has been given to their practicality in a production setting. To overcome these challenges, the authors propose a dual encoder architecture that enables rapid inference and scales effectively with the size of the whitelist. Additionally, they introduce and compare two methods for generating whitelists and conduct a comprehensive analysis of both the model and whitelists. The experimental results on a large proprietary help desk chat dataset demonstrate production-quality performance through offline metrics and human evaluation. This research provides key lessons about conversation modeling in practice and emphasizes the importance of considering practicality when developing conversation systems. By addressing these challenges and providing insights into effective model architectures and whitelist generation methods, this study contributes to building more efficient and reliable retrieval-based chatbots for real-world applications.