# TrustLLM: Trustworthiness in Large Language Models

**Authors:** Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bhavya Kailkhura, Caiming Xiong, Chao Zhang, Chaowei Xiao, Chunyuan Li, Eric Xing  
**arXiv:** [2401.05561](https://arxiv.org/abs/2401.05561)  
**Date:** 2024-01-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.05561v1/

## Key points

- Large language models (LLMs) like ChatGPT have impressive natural language processing capabilities
- Trustworthiness of LLMs is a crucial focus area
- TrustLLM introduces principles for trustworthy LLMs spanning eight dimensions
- TrustLLM establishes benchmarks across six key dimensions: truthfulness, safety, fairness, robustness, privacy, and machine ethics
- Positive correlation between trustworthiness and utility in LLMs
- Proprietary LLMs generally outperform open-source counterparts in trustworthiness
- Transparency is critical for trustworthiness in LLMs

## Summary

, , , , 

Large language models (LLMs) like ChatGPT have garnered significant attention for their impressive natural language processing capabilities. However, these LLMs also pose challenges, particularly in terms of trustworthiness. As a result, ensuring the trustworthiness of LLMs has become a crucial area of focus. This paper introduces TrustLLM, a comprehensive study that delves into the various dimensions of trustworthiness in LLMs. The study begins by outlining a set of principles for trustworthy LLMs that span eight different dimensions. Building upon these principles, TrustLLM establishes a benchmark across six key dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. The evaluation involves analyzing 16 mainstream LLMs using over 30 datasets. The findings from TrustLLM reveal several important insights. Firstly, there is a positive correlation between trustworthiness and utility (functional effectiveness) in LLMs. Additionally, proprietary LLMs generally outperform open-source counterparts in terms of trustworthiness, raising concerns about the risks associated with widely accessible open-source models. However, some open-source LLMs demonstrate comparable performance to proprietary ones. Furthermore, the study highlights instances where certain LLMs may prioritize exhibiting trustworthiness to such an extent that they compromise their utility by incorrectly flagging benign prompts as harmful and failing to respond appropriately. Transparency emerges as a critical factor not only within the models themselves but also in the underlying technologies that contribute to trustworthiness. In conclusion, TrustLLM underscores the importance of transparency and accountability in ensuring the trustworthiness of large language models. By identifying key principles and benchmarks for evaluating trustworthiness across various dimensions, this study provides valuable insights into enhancing the reliability and effectiveness of LLMs while addressing potential ethical concerns and challenges in their deployment.