# Enriching Conversation Context in Retrieval-based Chatbots

**Authors:** Amir Vakili Tahami, Azadeh Shakery  
**arXiv:** [1911.02290](https://arxiv.org/abs/1911.02290)  
**Date:** 2019-11-06  
**Summary page:** https://www.summarizepaper.com/en/paper/1911.02290v1/

## Key points

- The paper discusses retrieval-based chatbots and their classification into Cross-encoders and Bi-encoders
- Cross-encoders perform word matching over the pair, while Bi-encoders encode the pair separately
- Bi-encoders have better performance but are slower due to offline encoding of candidate responses
- Pre-trained language models have been effective in various natural language processing tasks
- Recent research shows that language models can be used to create Bi-encoders with faster inference speed
- The authors develop a sequence matching architecture that considers contexts in the training dataset at inference time
- They use the entire training set as a makeshift knowledge base during inference
- Experiments demonstrate that this architecture improves Bi-encoder performance while maintaining high inference speed
- The approach enhances retrieval-based chatbots by incorporating conversation context and leveraging pre-trained language models

## Summary

In the paper "Enriching Conversation Context in Retrieval-based Chatbots" by Amir Vakili Tahami and Azadeh Shakery, the authors discuss the work on retrieval-based chatbots and how it can be divided into two types: Cross-encoders and Bi-encoders. Cross-encoders perform word matching over the pair, while Bi-encoders encode the pair separately. Although Bi-encoders have better performance, they are slower because candidate responses cannot be encoded offline. The authors highlight that multi-layer transformer architectures pre-trained as language models have been effective in various natural language processing and information retrieval tasks. Recent research has shown that these language models can be used to create Bi-encoders that perform almost as well as Cross-encoders but with faster inference speed. Building upon this previous work, the authors develop a sequence matching architecture that takes into account contexts in the training dataset at inference time. They utilize the entire training set as a makeshift knowledge base during inference. Through detailed experiments, they demonstrate that this architecture further improves Bi-encoder performance while maintaining a relatively high inference speed. Overall, this paper presents an approach to enhance retrieval-based chatbots by incorporating conversation context and leveraging pre-trained language models. The findings suggest that this method can effectively improve performance without sacrificing efficiency in terms of inference speed.