# A Survey on Evaluation of Large Language Models

**Authors:** Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, Xing Xie  
**arXiv:** [2307.03109](https://arxiv.org/abs/2307.03109)  
**Date:** 2023-07-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.03109v1/

## Key points

- Large language models (LLMs) have remarkable capabilities in natural language processing tasks
- The authors provide a comprehensive review of evaluation methods for LLMs
- Three key dimensions of evaluation: what to evaluate, where to evaluate, and how to evaluate
- Overview of evaluation tasks in various areas such as reasoning, medical usage, ethics, education, natural and social sciences, agent applications, etc.
- Discussion on evaluation methods and benchmarks used in assessing LLM performance
- Success and failure cases of LLMs in different tasks
- Importance of treating evaluation as an essential discipline for developing more proficient LLMs
- Discussion on future challenges in LLMs evaluation

## Summary

Large language models (LLMs) have gained significant attention in recent years due to their remarkable capabilities in natural language processing tasks. In this paper, the authors provide a comprehensive review of evaluation methods for LLMs, focusing on three key dimensions: what to evaluate, where to evaluate, and how to evaluate. They first provide an overview of evaluation tasks including general natural language processing tasks, reasoning, medical usage, ethics, education, natural and social sciences, agent applications and other areas. Then they dive into the evaluation methods and benchmarks used in assessing the performance of LLMs. The paper also discusses success and failure cases of LLMs in different tasks and highlights the importance of treating evaluation as an essential discipline to aid the development of more proficient LLMs. The paper concludes with a discussion on future challenges in LLMs evaluation.