# On Early Detection of Hallucinations in Factual Question Answering

**Authors:** Ben Snyder, Marius Moisescu, Muhammad Bilal Zafar  
**arXiv:** [2312.14183](https://arxiv.org/abs/2312.14183)  
**Date:** 2023-12-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.14183v3/

## Key points

- Authors: Ben Snyder, Marius Moisescu, Muhammad Bilal Zafar
- Challenge: Hallucinations in large language models (LLMs) hinder user trust despite advancements
- Investigation: Can artifacts in model generations indicate potential hallucinations?
- Methodology: Analyzing LLMs at different stages (inputs, outputs, internal states)
- Findings: Differences in artifact distributions between hallucinated and non-hallucinated model generations
- Solution: Development of binary classifiers using artifacts as input features to classify model outputs
- Results: Hallucination classifiers achieve AUROC score of up to 0.80
- Insight: Tokens preceding a hallucination can predict subsequent occurrence
- Impact: Early detection capability enhances understanding and addressing of hallucinations within factual question answering contexts

## Summary

, , , , 


In their paper titled "On Early Detection of Hallucinations in Factual Question Answering," authors Ben Snyder, Marius Moisescu, and Muhammad Bilal Zafar address the challenge of hallucinations in large language models (LLMs) that hinder user trust despite the models' advancements in assisting with various tasks. The authors highlight that the fluency and coherence of LLM-generated content, even when containing hallucinations, pose difficulties in detection. To tackle this issue, they investigate whether artifacts present in model generations can serve as indicators of potential hallucinations. By analyzing LLMs at different stages - including inputs through Integrated Gradients based token attribution, outputs via Softmax probabilities, and internal states through self-attention and fully-connected layer activations - the researchers aim to identify signs of hallucinations specifically in open-ended question answering tasks. Their findings reveal distinct differences in artifact distributions between hallucinated and non-hallucinated model generations, suggesting a potential avenue for distinguishing between the two. Leveraging this insight, the authors develop binary classifiers that utilize these artifacts as input features to classify model outputs into hallucinatory and non-hallucinatory categories. Remarkably, these hallucination classifiers achieve an impressive AUROC score of up to 0.80, underscoring their effectiveness in identifying problematic content generated by LLMs. Moreover, the study demonstrates that tokens preceding a hallucination can predict the occurrence of a subsequent hallucination even before it manifests. This early detection capability offers valuable insights into understanding and addressing hallucinations within factual question answering contexts. Overall, this research sheds light on innovative approaches for enhancing user trust in LLM-generated content by proactively identifying and mitigating instances of misinformation or inaccuracies caused by hallucinations.