# Pre-trained language models as knowledge bases for Automotive Complaint Analysis

**Authors:** V. D. Viellieber, M. Aßenmacher  
**arXiv:** [2012.02558](https://arxiv.org/abs/2012.02558)  
**Date:** 2020-12-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2012.02558v1/

## Key points

- Authors Viellieber and Aßenmacher explore the use of pre-trained language models like BERT for storing factual knowledge extracted from their pre-training corpus.
- The research focuses on evaluating these models in identifying technical quality issues in unstructured customer feedback within the automotive industry.
- The researchers designed domain-specific probes to assess the models' ability to recognize topics related to automotive complaints.
- Experiments showed promising results, with most architectures achieving Precision@1 scores exceeding 60% and high accuracy levels for P@5 and P@10 metrics, reaching up to 90% in some cases.
- The study highlights the potential of using advanced language models as comprehensive knowledge bases for structured analysis of customer feedback, particularly in industries like automotive.

## Summary

In their research titled "Pre-trained language models as knowledge bases for Automotive Complaint Analysis," authors V. D. Viellieber and M. Aßenmacher delve into the capabilities of large pre-trained language models such as BERT to store factual knowledge extracted from their pre-training corpus. Building upon previous findings by Petroni et al. (2019), the authors focus on evaluating these models in the context of identifying technical quality issues within unstructured customer feedback in the automotive industry. To assess the ability of pre-trained models to recognize domain-specific topics related to automotive complaints, the researchers designed a series of probes tailored for this purpose. By employing fill-in-the-mask tasks and continually pre-training the models on data from the Office of Defects Investigation (ODI) Complaints dataset, they aimed to enhance the models' understanding and performance in this specific domain. The experiments conducted by Viellieber and Aßenmacher revealed promising results, with most evaluated architectures achieving a Precision@1 (P@1) score exceeding 60%. Notably, for P@5 and P@10 metrics, accuracy levels soared well above 80% and even reached up to 90% in some instances. These outcomes underscore the potential utility of leveraging language models as comprehensive knowledge bases for structured analysis of customer feedback within industries like automotive. Overall, this study sheds light on how advanced language models can be effectively harnessed to extract valuable insights from unstructured textual data, offering new avenues for enhancing customer feedback analysis processes in specialized domains such as automotive complaints.