# Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics

**Authors:** Martin Boyanov, Ivan Koychev, Preslav Nakov, Alessandro Moschitti, Giovanni Da San Martino  
**arXiv:** [1710.00689](https://arxiv.org/abs/1710.00689)  
**Date:** 2017-10-02  
**Summary page:** https://www.summarizepaper.com/en/paper/1710.00689v1/

## Key points

- The paper proposes a novel approach to training chatbots using question answering (QA) data from web forums.
- Pairs of question and answer sentences are extracted from longer texts in forum discussions.
- Shorter texts are used to train sequence-to-sequence (seq2seq) models more efficiently.
- A new model selection strategy based on QA measures is introduced to improve parameter optimization.
- Extrinsic evaluation with respect to a QA task is suggested as an automatic evaluation method for chatbots.
- The proposed model achieves a Mean Average Precision (MAP) of 63.5% on the extrinsic task.
- The chatbot can accurately answer 49.5% of questions similar to those asked in the forum and 47.3% of more conversational questions.
- This research presents an innovative approach for training chatbots using QA data from web forums and introduces new strategies for model selection and evaluation.

## Summary

The paper "Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics" proposes a novel approach to training chatbots using question answering (QA) data from web forums. The authors begin by extracting pairs of question and answer sentences from the longer texts found in forum discussions. These shorter texts are then used to train sequence-to-sequence (seq2seq) models in a more efficient manner. To improve parameter optimization, the authors introduce a new model selection strategy based on QA measures. This strategy helps identify the most effective models for chatbot training. In addition, the paper suggests using extrinsic evaluation with respect to a QA task as an automatic evaluation method for chatbots. The evaluation results show that the proposed model achieves a Mean Average Precision (MAP) of 63.5% on the extrinsic task. Furthermore, the chatbot demonstrates its ability to correctly answer questions in different styles. It can accurately answer 49.5% of questions similar to those asked in the forum and 47.3% of more conversational questions. Overall, this research presents an innovative approach for training chatbots using QA data from web forums and introduces new strategies for model selection and evaluation that enable improved parameter optimization and accurate automatic evaluation results.