# Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT

**Authors:** Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao  
**arXiv:** [2302.10198](https://arxiv.org/abs/2302.10198)  
**Date:** 2023-02-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.10198v2/

## Key points

- The study explores the understanding ability of ChatGPT.
- ChatGPT has remarkable generation ability compared to existing models but little attention has been given to its quantitative analysis of understanding ability.
- The authors evaluate ChatGPT's performance on the most popular GLUE benchmark and compare it with four representative fine-tuned BERT-style models.
- ChatGPT generates fluent and high-quality responses to human inquiries but falls short in handling paraphrase and similarity tasks.
- It outperforms all BERT models on inference tasks by a large margin.
- ChatGPT achieves comparable performance compared with BERT on sentiment analysis and question-answering tasks.
- Advanced prompting strategies such as CoT can enhance its performance significantly.
- Further research is needed to develop more sophisticated techniques for improving its performance in handling paraphrase and similarity tasks.

## Summary

The study titled "Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT" by Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao explores the understanding ability of ChatGPT. While several prior studies have shown that ChatGPT attains remarkable generation ability compared with existing models, its quantitative analysis of its understanding ability has been given little attention. To address this gap, the authors evaluate ChatGPT's performance on the most popular GLUE benchmark and compare it with four representative fine-tuned BERT-style models. The findings show that while ChatGPT generates fluent and high-quality responses to human inquiries, it falls short in handling paraphrase and similarity tasks. However, it outperforms all BERT models on inference tasks by a large margin. Additionally, ChatGPT achieves comparable performance compared with BERT on sentiment analysis and question-answering tasks. To further improve the understanding ability of ChatGPT, the authors combine some advanced prompting strategies such as CoT. The results demonstrate that these strategies can enhance its performance significantly. Overall, this study provides insights into the strengths and limitations of ChatGPT in terms of its understanding ability. It highlights the need for further research to develop more sophisticated techniques for improving its performance in handling paraphrase and similarity tasks.