# In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT

**Authors:** Xinyue Shen, Zeyuan Chen, Michael Backes, Yang Zhang  
**arXiv:** [2304.08979](https://arxiv.org/abs/2304.08979)  
**Date:** 2023-04-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.08979v2/

## Key points

- Researchers conducted a large-scale measurement of ChatGPT's reliability in generic question-answering scenarios
- 5,695 questions were curated across ten datasets and eight domains to assess ChatGPT's performance
- ChatGPT's reliability varied across different domains, with notable underperformance in law and science questions
- System roles designed by OpenAI could subtly impact ChatGPT's reliability
- ChatGPT was vulnerable to adversarial examples where even a single character change could negatively affect its performance
- Evaluation pipeline consisted of four steps: query formation, ChatGPT invocation, answer extraction, and evaluation
- BERTopic was used for automatic topic modeling to categorize questions into broad domains; GTR-T5-XL was selected based on high CV coherence score
- Data samples were categorized into history, law, general works, medicine, social science, science, technology and recreation domains through manual inspection and a priori coding approach
- Study included statistics of QA datasets such as yes/no QA (YN), multiple-choice QA (MC), extractive QA (EX), abstractive QA (AB) with varying numbers of questions and evaluation metrics
- Research provides insights into ChatGPT's reliability across different domains and emphasizes the importance of enhancing the reliability and security of large language models like ChatGPT.

## Summary

In this study, the researchers conducted a large-scale measurement of ChatGPT's reliability in generic question-answering scenarios. They carefully curated 5,695 questions across ten datasets and eight domains to assess ChatGPT's performance. The results showed that ChatGPT's reliability varied across different domains, with notable underperformance in law and science questions. The researchers also discovered that system roles designed by OpenAI could subtly impact ChatGPT's reliability. Additionally, they found that ChatGPT was vulnerable to adversarial examples where even a single character change could negatively affect its performance. The evaluation pipeline consisted of four steps: query formation, ChatGPT invocation, answer extraction, and evaluation. This workflow is illustrated in Figure 1. To categorize the questions into broad domains, the researchers used BERTopic for automatic topic modeling and applied deductive analysis. They tested five pre-trained embedding models for BERTopic and selected GTR-T5-XL based on its high CV coherence score. Through manual inspection and a priori coding approach, the data samples were categorized into history, law, general works, medicine, social science, science, technology and recreation domains. While acknowledging potential underrepresentation of certain domains due to their coding procedure,the researchers ensured adequate representation in the study by utilizing representative scores to filter out unclearly classified questions during topic modeling. The study included statistics of QA datasets such as yes/no QA (YN), multiple-choice QA (MC), extractive QA (EX), abstractive QA (AB) with varying numbers of questions and evaluation metrics. Overall,this research provides valuable insights into ChatGPT's reliability across different domains and highlights the importance of enhancing the reliability and security of large language models like ChatGPT.