Running summarizing tools on a new article

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a prominent paradigm for enhancing the factual grounding of large language models (LLMs) by incorporating external knowledge into the generation process. Despite these advances, hallucinations remain a significant challenge, where models confidently produce information that is factually incorrect or unsupported. In this work, we conduct a comprehensive empirical analysis of the relationship between model confidence, semantic fidelity, and hallucination patterns across three diverse datasets: XSum, TruthfulQA, and SciFact. Through systematic evaluation of ROUGE, BERTScore, perplexity, entropy, and semantic similarity metrics, we reveal critical insights into how dataset characteristics influence hallucination risk. Our analysis demonstrates that low entropy paired with poor semantic alignment signals overconfident hallucinations, particularly in abstractive summarization tasks, while high entropy with strong semantic fidelity indicates cautious, faithful generation in retrieval-grounded scenarios. We identify significant effect sizes across datasets and provide comprehensive visualizations that illustrate metric interdependencies and hallucination patterns. Our findings establish that hallucination behavior is multifactorial, influenced by task abstraction level, retrieval grounding, and domain complexity. This work provides empirical foundations for understanding confidence-fidelity dynamics and informs the design of future adaptive hallucination mitigation strategies in trustworthy LLM deployment.