# Submodularity-Inspired Data Selection for Goal-Oriented Chatbot Training Based on Sentence Embeddings

**Authors:** Mladen Dimovski, Claudiu Musat, Vladimir Ilievski, Andreea Hossmann, Michael Baeriswyl  
**arXiv:** [1802.00757](https://arxiv.org/abs/1802.00757)  
**Date:** 2018-02-02  
**Summary page:** https://www.summarizepaper.com/en/paper/1802.00757v2/

## Key points

- The paper addresses challenges faced by spoken language understanding (SLU) systems, such as goal-oriented chatbots or personal assistants
- SLU systems often require a large amount of in-domain training data, leading to data availability issues
- The authors propose a technique called data selection in the low-data regime to overcome this problem
- The key idea is to use a submodularity-inspired data ranking function called the ratio-penalty marginal gain
- This function selects data points for labeling based solely on information extracted from the textual embedding space
- The authors compare their method with two known active learning techniques and show that it outperforms them
- Their proposed selection technique does not require retraining the model between selection steps, making it time-efficient
- By leveraging textual embeddings and utilizing submodularity-inspired ranking, this approach provides an effective solution for training SLU systems with limited labeled data.

## Summary

The paper titled "Submodularity-Inspired Data Selection for Goal-Oriented Chatbot Training Based on Sentence Embeddings" addresses the challenges faced by spoken language understanding (SLU) systems, such as goal-oriented chatbots or personal assistants. These systems rely on a natural language understanding (NLU) module to determine user intent and extract relevant information from user queries. However, SLU systems often require a large amount of in-domain training data, leading to data availability issues that hinder system development. To overcome this problem, the authors propose a technique called data selection in the low-data regime. This technique allows training with fewer labeled sentences, reducing labeling costs. The key idea is to use a submodularity-inspired data ranking function called the ratio-penalty marginal gain. This function selects data points for labeling based solely on information extracted from the textual embedding space. The authors demonstrate that distances in the embedding space can serve as a viable source of information for data selection. They compare their method with two known active learning techniques and show that it outperforms them, enabling cost-efficient training of the NLU unit. One notable advantage of their proposed selection technique is that it does not require retraining the model between selection steps, making it time-efficient. By leveraging textual embeddings and utilizing submodularity-inspired ranking, this approach provides an effective solution for training SLU systems with limited labeled data. In conclusion, this paper presents a novel approach to address data availability issues in SLU systems by proposing a submodularity-inspired data selection technique based on sentence embeddings. The results demonstrate its superiority over existing methods and highlight its potential for cost-efficient training of NLU units in goal-oriented chatbots or personal assistants. In summary, this paper proposes an innovative solution to tackle data availability issues in SLU systems through submodularity inspired data selection based on sentence embeddings which outperforms existing methods while being time efficient and cost effective for training NLUs in goal oriented chatbots or personal assistants.