# Distinguishing Ignorance from Error in LLM Hallucinations

**Authors:** Adi Simhi, Jonathan Herzig, Idan Szpektor, Yonatan Belinkov  
**arXiv:** [2410.22071](https://arxiv.org/abs/2410.22071)  
**Date:** 2024-10-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.22071v1/

## Key points

- Large language models (LLMs) generate hallucinations that are unsupported, factually inaccurate, or inconsistent with previous data
- Differentiation between two types of hallucinations: lack of correct answer within model's parameters vs. incorrect response despite possessing necessary knowledge is crucial
- Proposed interventions within model's internal computation to address incorrect answers and reliance on external knowledge sources when correct answer is not present in parameters
- Introduction of novel approach called Wrong Answer despite having Correct Knowledge (WACK) to distinguish between types of hallucinations
- Probing experiments demonstrate distinct manifestations of hallucinations in inner states of the model and variations across models even with common factual knowledge
- Training a probe on specialized WACK datasets more effective in detecting case (2) hallucinations compared to generic datasets commonly used
- Study highlights nuanced nature of LLM hallucinations and offers insights into mitigating them through targeted dataset construction and probing techniques

## Summary

In their research titled "Distinguishing Ignorance from Error in LLM Hallucinations," authors Adi Simhi, Jonathan Herzig, Idan Szpektor, and Yonatan Belinkov delve into the challenges posed by large language models (LLMs) in generating hallucinations. These are outputs that are unsupported, factually inaccurate, or inconsistent with previous data. The study specifically focuses on close-book Question Answering (CBQA) and highlights a critical gap in existing literature concerning the differentiation between two types of hallucinations: those where the model lacks the correct answer within its parameters and those where it provides an incorrect response despite possessing the necessary knowledge. The researchers argue that discerning between these two scenarios is essential for effectively identifying and addressing hallucinations. They propose interventions within the model's internal computation to mitigate instances where incorrect answers are generated despite having access to relevant information stored in its parameters. In contrast, when the correct answer is not present within the model's parameters, external knowledge sources or refraining from providing an answer altogether may be necessary for resolution. To aid in distinguishing between these two types of hallucinations, the authors introduce a novel approach called Wrong Answer despite having Correct Knowledge (WACK). This method involves constructing model-specific datasets tailored to detect instances of hallucinations stemming from a lack of parametric knowledge. Through probing experiments, they demonstrate that these distinct types of hallucinations manifest differently in the inner states of the model. Furthermore, by analyzing datasets generated using WACK across various models, the researchers reveal that even when models share common factual knowledge, they exhibit variations in specific examples leading to hallucinatory outputs. Finally, training a probe on these specialized WACK datasets proves to be more effective in detecting case (2) hallucinations compared to using generic one-size-fits-all datasets commonly employed for this purpose. Overall, this study sheds light on the nuanced nature of LLM hallucinations and offers valuable insights into mitigating these phenomena through targeted dataset construction and probing techniques. The research findings provide a foundation for improving detection mechanisms and enhancing overall performance in close-book Question Answering systems plagued by hallucinatory outputs.