# How to Build Robust FAQ Chatbot with Controllable Question Generator?

**Authors:** Yan Pan, Mingyang Ma, Bernhard Pflugfelder, Georg Groh  
**arXiv:** [2112.03007](https://arxiv.org/abs/2112.03007)  
**Date:** 2021-11-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.03007v1/

## Key points

- Challenges of building a robust FAQ chatbot
- Proposal of diversity controllable semantically valid adversarial attacker (DCSA) method
- Generation of high-quality and diverse question-answer pairs
- Successful fooling of passage retrieval model with generated QA pairs
- Study on robustness and generalization of QA model with generated data set
- Improved generalizability to new domains and ability to detect unanswerable adversarial questions
- Use of semantic and syntactic filters to sample valuable adversarial triples from unstructured text
- Analysis of generated samples from semantic, syntactic, and fluency aspects
- Benefits of proposed method in terms of generalization and robustness across different domains
- Contribution statement by authors: Yan Pan, Mingyang Ma, Bernhard Pflugfelder, Georg Groh
- Use of multiple source datasets to improve performance and robust generalization of QA models
- Effectiveness of reading comprehension models combined with search components for question answering tasks
- Highlighting the use of TF-IDF/BM25 retrieval systems
- Overall system architecture for generating diverse questions using a semantic graph:
- Dataset sampler for recognizing facts and relationships as symbolic presentations with a semantic graph
- High-quality question generation model fine-tuned on constructed data
- Question filters based on semantic and syntactic features
- FAQ chatbot to evaluate quality of adversarial examples
- Explanation of parsing answer style and clues for question generation
- Mining candidate facts from passages using SceneGraphParser
- Selection of multiple clues and answers from semantic graph
- Evaluation of relationship between clues over graph for semantic consistency
- Discussion on GPT2 based question generation method and its power in generating diverse samples
- Emphasis on ACS aware question generation model with semantic control over GPT2

## Summary

The existing summary discusses the challenges of building a robust FAQ chatbot and proposes a method called diversity controllable semantically valid adversarial attacker (DCSA) to generate high-quality and diverse question-answer pairs. The generated QA pairs successfully fool the passage retrieval model, and a study is conducted to analyze the robustness and generalization of the QA model with the generated data set. The results show that the generated data set improves the generalizability of the QA model to new domains and enhances its ability to detect unanswerable adversarial questions. In addition to the existing summary, further context is provided. The content mentions the use of semantic and syntactic filters to sample valuable adversarial triples from unstructured text. It also highlights the analysis of generated samples from semantic, syntactic, and fluency aspects. Compared to existing question generation methods, the proposed method demonstrates benefits in terms of generalization and robustness across different domains. The contribution statement reveals that Yan Pan contributed to conceptualization, methodology, validation, formal investigation, visualization, project administration, and writing-original draft; Mingyang Ma contributed to conceptualization, validation, methodology supervision administration and writing-review & editing; Bernhard Pflugfelder contributed to conceptualization methodology writing-review & editing supervision project administration and funding acquisition; Georg Groh contributed to conceptualization writing-review & editing project administration supervisionand project management. Furthermore ,the content discusses how multiple source datasets can improve the performance and robust generalization of QA models. It mentions that reading comprehension models combined with search components can effectively handle question answering tasks. The use of TF-IDF/BM25 retrieval systems is also highlighted. The methodology section describes an overall system architecture for generating diverse questions using a semantic graph which consists of four components: dataset sampler for recognizing facts and relationships as symbolic presentations with a semantic graph; high-quality question generation model fine-tuned on constructed data; question filters based on semantic and syntactic features;  and an FAQ chatbot to evaluate quality of adversarial examples .The parsing of answer style  and clues for question generation is explained .The sampler mines candidate facts from passages using SceneGraphParser  and selects multiple clues  and answers from semantic graph .The relationship between clues over graph is evaluated  to ensure semantic consistency .The GPT2 based question generation method is discussed highlighting its power in generating diverse samples .The use of ACS aware question generation model with semantic control over GPT2 is emphasized . Overall ,the refined detailed longer summary provides comprehensive overview about proposed method for generating high quality  and diverse question answer pairs using semantic graph .It also discusses analysis of generated samples ,contribution statement by authors ,benefits of multiple source datasets ,use GPT2 based question generation .