# Overconfident and Unconfident AI Hinder Human-AI Collaboration

**Authors:** Jingshu Li, Yitian Yang, Renwen Zhang, Yi-chieh Lee  
**arXiv:** [2402.07632](https://arxiv.org/abs/2402.07632)  
**Date:** 2024-02-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.07632v3/

## Key points

- Transparency is crucial in AI to ensure responsible deployment and effective collaboration between humans and AI systems.
- Communication of uncertainty, such as providing users with information about AI's confidence level, is essential for transparency.
- Uncalibrated confidence levels in AI can lead to overestimation or underestimation of correctness likelihood, posing risks to human-AI collaboration.
- A study by Jingshu Li et al. explores the impact of uncalibrated AI confidence on trust, adoption of advice, and overall collaboration outcomes.
- Lack of trust calibration support exacerbates issues related to uncalibrated confidence and can lead to misuse or underutilization of AI capabilities.
- Introducing trust calibration support can help identify instances of uncalibration and reduce misuse tendencies but may also foster distrust among users.
- Calibrating AI confidence levels is critical for optimizing human-AI collaboration outcomes and future design practices should focus on promoting transparent interactions.

## Summary

In the realm of artificial intelligence (AI), transparency plays a crucial role in ensuring responsible deployment and fostering effective collaboration between humans and AI systems. One key aspect of transparency is the communication of uncertainty, such as providing users with information about AI's confidence level or correctness likelihood (CL). However, a common challenge arises from the fact that these confidence levels are often uncalibrated, leading to potential overestimation or underestimation of the actual CL. This lack of calibration poses risks and potential harms to human-AI collaboration. A recent study conducted by Jingshu Li, Yitian Yang, Renwen Zhang, and Yi-chieh Lee delves into the impact of uncalibrated AI confidence on various aspects of human-AI interaction. The research specifically focuses on how uncalibrated confidence levels influence users' trust in AI, their adoption of AI advice, and overall collaboration outcomes. Additionally, the study explores how increased transparency through trust calibration support can mitigate these effects. The findings from this investigation shed light on the detrimental consequences of uncalibrated AI confidence. It was discovered that when AI exhibits overconfident behavior, users tend to misuse its recommendations. Conversely, when AI displays low confidence levels, users may disregard its input altogether. These behaviors ultimately hinder the success of human-AI collaboration efforts. Moreover, the study highlights that a deficiency in trust calibration support exacerbates issues related to uncalibrated confidence. Without proper mechanisms in place to detect and address calibration discrepancies, users are more likely to fall into patterns of misusing or underutilizing AI capabilities. On the other hand, introducing trust calibration support can help identify instances of uncalibration and reduce misuse tendencies. However,this enhanced transparency also has its drawbacks - it may inadvertently foster distrust among users and lead to decreased reliance on AI-generated insights. Overall,the findings underscore the critical importance of calibrating AI confidence levels for optimizing human-AI collaboration outcomes. The study suggests valuable directions for future AI design practices and regulatory frameworks aimed at promoting transparent and trustworthy interactions between humans and intelligent systems.