RAG et hallucinations des grands modèles de langage

In short

Les résultats disponibles ne permettent pas d’affirmer que le RAG réduit systématiquement davantage les hallucinations que le fine-tuning seul : les comparaisons directes sont peu nombreuses et portent sur des formes différentes d’adaptation et d’évaluation. [12][13][17][24] Une expérience publiée favorise le RAG face au fine-tuning non supervisé pour intégrer des connaissances, mais une autre étude trouve de meilleures performances avec le modèle ajusté ; ces résultats concernent surtout la réussite aux tâches, qui ne mesure pas exclusivement les hallucinations. [12][17] En biomédecine, une méta-analyse rapporte un bénéfice global du RAG face aux modèles de base, sans établir sa supériorité sur le fine-tuning seul, tandis qu’une revue de sept études favorise les systèmes hybrides FT + RAG. [20][24] Le bénéfice dépend fortement de la pertinence des sources, de leur utilisation fidèle et du protocole d’évaluation ; les méthodes adaptatives, la vérification et l’apprentissage spécifique à l’ancrage documentaire cherchent précisément à corriger les défaillances du RAG simple. [6][8][15][23][30]

This summary rests on the findings below, each with the sentences that support it.

Approaches compared

Where the papers agree

• L’accès à des documents ne suffit pas : il faut à la fois retrouver des preuves pertinentes et en préserver fidèlement le contenu pendant la génération. [7][23]
  2 supporting sentences

  “LMs trained without retrieval and attribution in mind does not always generate sentences that can be attributed to in-context evidence documents, even when provided relevant documents only.” [7] · full text

  “Second, grounded generation requires both relevant evidence and faithful use of that evidence.” [23] · full text

• La combinaison du RAG et d’un apprentissage adapté apparaît prometteuse, mais les résultats favorables ne constituent pas une garantie générale de sécurité ou d’absence d’hallucinations. [13][24]
  2 supporting sentences

  “Our results show that the hybrid approach using both RAG and fine-tuning performs best in CRAG.” [13] · full text

  “Future work should prioritize standardized evaluation, workflow integration, and governance to enable safe deployment.” [24]

What they leave open

• Il manque une comparaison harmonisée du RAG, du fine-tuning seul et de leur combinaison, à modèle, budget, corpus et mesure directe des hallucinations comparables ; les classements divergents disponibles ne constituent donc pas une contradiction expérimentale établie. [12][13][17][24]
  No paper states it in these words: inferred from what the papers do not study. Check it.
• La transférabilité aux conditions réelles reste ouverte, notamment lorsque les tâches financières dépassent celles testées ou que de nombreux documents de qualité variable remplacent des paires synthétiques simples. [2][18]
  2 supporting sentences

  “Firstly, our tasks, although representative, cannot encompass the full spectrum of real-world tasks in the vast and varied domain of finance.” [2] · full text

  “This creates a simplified binary contrast that may not generalize to real-world scenarios where multiple retrieved documents vary in quality.” [18] · full text

• L’évaluation devrait distinguer systématiquement exactitude, fidélité aux sources, attribution et pertinence de la recherche, car la standardisation actuelle reste limitée. [22][23]
  2 supporting sentences

  “However, mitigation remains constrained by retrieval noise, latency, limited benchmark standardization, and weak integration between hallucination detection and correction.” [22]

  “Third, evaluation should move beyond answer accuracy toward factuality, faithfulness, attribution, and retrieval relevance.” [23] · full text

• La généralisation des méthodes entraînées de détection aux interactions prolongées et à d’autres capacités du modèle reste insuffisamment établie. [26]
  1 supporting sentence

  “As a limitation of the current study, we focused on single-turn interactions and on one task (Q&A).” [26] · full text

References

1. [1] Retrieval Augmentation Reduces Hallucination in Conversation

   Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela et al. · 2021 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · cited 739 times · doi:10.18653/v1/2021.findings-emnlp.320

   Founding Récupération pour dialogue ancré · experiment

   Why it is here: Évalue la récupération dans le dialogue et ses effets sur la connaissance et la qualité, sans comparaison directe au fine-tuning.

   Abstract

   Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2021).In this work we explore the use of neural-retrieval-in-the-loop architectures -recently shown to be effective in open-domain QA (Lewis et al., 2020b; Izacard and Grave, 2021b) -for knowledge-grounded dialogue, a task that is arguably more challenging as it requires querying based on complex multi-turn dialogue context and generating conversationally coherent responses.We study various types of architectures with multiple components -retrievers, rankers, and encoder-decoders -with the goal of maximizing knowledgeability while retaining conversational ability.We demonstrate that our best models obtain state-of-the-art performance on two knowledge-grounded conversational tasks.The models exhibit open-domain conversational capabilities, generalize effectively to scenarios not within the training data, and, as verified by human evaluations, substantially reduce the well-known problem of knowledge hallucination in state-of-the-art chatbots. * Equal ContributionThe following is a conversation with an AI assistant.The assistant is helpful, creative, clever, and very friendly.Human: Hello, who are you?AI: I am an AI created by OpenAI.How can I help you today?Human: Tell me about Kyunghyun

2. [2] Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination

   Haoqiang Kang, Xiao-Yang Liu · 2023 · RePEc: Research Papers in Economics preprint · cited 67 times · doi:10.48550/arxiv.2311.15548

   Preprint Évaluation de méthodes financières · experiment

   Why it is here: Compare plusieurs méthodes de réduction des hallucinations en finance, dont le RAG, mais l’abstract ne précise pas une comparaison au fine-tuning.

   Abstract · conclusions of the full text read too

   The hallucination issue is recognized as a fundamental deficiency of large language models (LLMs), especially when applied to fields such as finance, education, and law. Despite the growing concerns, there has been a lack of empirical investigation. In this paper, we provide an empirical examination of LLMs' hallucination behaviors in financial tasks. First, we empirically investigate LLM model's ability of explaining financial concepts and terminologies. Second, we assess LLM models' capacity of querying historical stock prices. Third, to alleviate the hallucination issue, we evaluate the efficacy of four practical methods, including few-shot learning, Decoding by Contrasting Layers (DoLa), the Retrieval Augmentation Generation (RAG) method and the prompt-based tool learning method for a function to generate a query command. Finally, our major finding is that off-the-shelf LLMs experience serious hallucination behaviors in financial tasks. Therefore, there is an urgent need to call for research efforts in mitigating LLMs' hallucination.

3. [3] Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy

   Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang et al. · 2023 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · cited 458 times · doi:10.18653/v1/2023.findings-emnlp.620

   récupération-génération itérative · experiment

   Why it is here: Évalue une méthode de RAG itératif qui améliore la pertinence des preuves, mais ne la compare pas au fine-tuning seul.

   Abstract

   Retrieval-augmented generation has raise extensive attention as it is promising to address the limitations of large language models including outdated knowledge and hallucinations.However, retrievers struggle to capture relevance, especially for queries with complex information needs.Recent work has proposed to improve relevance modeling by having large language models actively involved in retrieval, i.e., to guide retrieval with generation.In this paper, we show that strong performance can be achieved by a method we call ITER-RETGEN, which synergizes retrieval and generation in an iterative manner: a model's response to a task input shows what might be needed to finish the task, and thus can serve as an informative context for retrieving more relevant knowledge which in turn helps generate a better response in another iteration.Compared with recent work which interleaves retrieval with generation when completing a single output, ITER-RETGEN processes all retrieved knowledge as a whole and largely preserves the flexibility in generation without structural constraints.We evaluate ITER-RETGEN on multi-hop question answering, fact verification, and commonsense reasoning, and show that it can flexibly leverage parametric knowledge and non-parametric knowledge, and is superior to or competitive with state-of-the-art retrieval-augmented baselines while causing fewer overheads of retrieval and generation.We can further improve performance via generation-augmented retrieval adaptation.

4. [4] HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models

   Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian‐Yun Nie et al. · 2023 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · cited 749 times · doi:10.18653/v1/2023.emnlp-main.397

   benchmark de détection d’hallucinations · method

   Why it is here: Fournit un benchmark et des mesures de détection utiles à l’évaluation, mais ne compare pas RAG et fine-tuning.

   Abstract

   Large language models (LLMs), such as Chat-GPT, are prone to generate hallucinations, i.e., content that conflicts with the source or cannot be verified by the factual knowledge.To understand what types of content and to which extent LLMs are apt to hallucinate, we introduce the Hallucination Evaluation benchmark for Large Language Models (HaluEval), a large collection of generated and human-annotated hallucinated samples for evaluating the performance of LLMs in recognizing hallucination.To generate these samples automatically, we propose a two-stage framework, i.e., samplingthen-filtering.Besides, we hire some human labelers to annotate the hallucinations in Chat-GPT responses.The empirical results suggest that ChatGPT is likely to generate hallucinated content related to specific topics by fabricating unverifiable information (i.e., about 19.5% responses).Moreover, existing LLMs face great challenges in recognizing the hallucinations in texts.However, our experiments also prove that providing external knowledge or adding reasoning steps can help LLMs recognize hallucinations.Our benchmark can be accessed at https://github.com/RUCAIBox/HaluEval.

5. [5] Retrieval-Augmented Generation for Large Language Models: A Survey

   Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia et al. · 2023 · arXiv (Cornell University) preprint · cited 6514 times · doi:10.48550/arxiv.2312.10997

   Review Preprint revue des architectures RAG · review

   Why it is here: Présente les variantes et composants du RAG pertinents pour expliquer les différences d’effet, mais pas une comparaison avec le fine-tuning.

   Abstract · conclusions of the full text read too

   Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes. Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases. This enhances the accuracy and credibility of the generation, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domain-specific information. RAG synergistically merges LLMs' intrinsic knowledge with the vast, dynamic repositories of external databases. This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG. It meticulously scrutinizes the tripartite foundation of RAG frameworks, which includes the retrieval, the generation and the augmentation techniques. The paper highlights the state-of-the-art technologies embedded in each of these critical components, providing a profound understanding of the advancements in RAG systems. Furthermore, this paper introduces up-to-date evaluation framework and benchmark. At the end, this article delineates the challenges currently faced and points out prospective avenues for research and development.

6. [6] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

   Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al. · 2023 · arXiv (Cornell University) preprint · cited 1091 times · doi:10.48550/arxiv.2310.11511

   Preprint Récupération adaptative et auto-critique · method

   Why it is here: Propose un RAG avec récupération à la demande et auto-évaluation, éclairant les mécanismes de grounding, sans comparaison au fine-tuning seul.

   Abstract

   Despite their remarkable capabilities, large language models (LLMs) often produce responses containing factual inaccuracies due to their sole reliance on the parametric knowledge they encapsulate. Retrieval-Augmented Generation (RAG), an ad hoc approach that augments LMs with retrieval of relevant knowledge, decreases such issues. However, indiscriminately retrieving and incorporating a fixed number of retrieved passages, regardless of whether retrieval is necessary, or passages are relevant, diminishes LM versatility or can lead to unhelpful response generation. We introduce a new framework called Self-Reflective Retrieval-Augmented Generation (Self-RAG) that enhances an LM's quality and factuality through retrieval and self-reflection. Our framework trains a single arbitrary LM that adaptively retrieves passages on-demand, and generates and reflects on retrieved passages and its own generations using special tokens, called reflection tokens. Generating reflection tokens makes the LM controllable during the inference phase, enabling it to tailor its behavior to diverse task requirements. Experiments show that Self-RAG (7B and 13B parameters) significantly outperforms state-of-the-art LLMs and retrieval-augmented models on a diverse set of tasks. Specifically, Self-RAG outperforms ChatGPT and retrieval-augmented Llama2-chat on Open-domain QA, reasoning and fact verification tasks, and it shows significant gains in improving factuality and citation accuracy for long-form generations relative to these models.

7. [7] Understanding Retrieval Augmentation for Long-Form Question Answering

   Hung-Ting Chen, Fangyuan Xu, Shane Arora, Eunsol Choi · 2023 · arXiv (Cornell University) preprint · cited 40 times · doi:10.48550/arxiv.2310.12150

   Preprint Études contrôlées de réponses longues · experiment

   Why it is here: Analyse l’attribution des réponses longues aux documents récupérés, une mesure importante du grounding, sans comparer au fine-tuning seul.

   Abstract · conclusions of the full text read too

   How retrieved documents are used in language models (LMs) for long-form generation task is understudied. We present two controlled studies on retrieval-augmented LM for long-form question answering (LFQA): one fixing the LM and varying evidence documents and the other fixing evidence documents and varying the LMs. We study various attributes of generated answers (e.g., fluency, length, variance), with an emphasis on the attribution of generated answers to in-context evidence documents. We collect a dataset (SALAD) containing human annotations of sentence-level answer attribution in LFQA and evaluate existing methods for automatically judging attribution. We find that while LMs can leverage relevant in-context documents, the generated answer is only partially attributable towards the documents, especially for LMs trained without retrieval augmentation. Together, our analysis reveals how retrieval augmentation impacts long knowledge-rich text generation and provide directions for future work.

8. [8] Benchmarking Large Language Models in Retrieval-Augmented Generation

   Jiawei Chen, Hongyu Lin, Xianpei Han, Le Sun · 2024 · Proceedings of the AAAI Conference on Artificial Intelligence · cited 755 times · doi:10.1609/aaai.v38i16.29728

   Recent benchmark des capacités RAG · method

   Why it is here: Le benchmark mesure la robustesse, le rejet et l’intégration d’information en RAG, éclairant la dépendance des résultats aux capacités évaluées.

   Abstract

   Retrieval-Augmented Generation (RAG) is a promising approach for mitigating the hallucination of large language models (LLMs). However, existing research lacks rigorous evaluation of the impact of retrieval-augmented generation on different large language models, which make it challenging to identify the potential bottlenecks in the capabilities of RAG for different LLMs. In this paper, we systematically investigate the impact of Retrieval-Augmented Generation on large language models. We analyze the performance of different large language models in 4 fundamental abilities required for RAG, including noise robustness, negative rejection, information integration, and counterfactual robustness. To this end, we establish Retrieval-Augmented Generation Benchmark (RGB), a new corpus for RAG evaluation in both English and Chinese. RGB divides the instances within the benchmark into 4 separate testbeds based on the aforementioned fundamental abilities required to resolve the case. Then we evaluate 6 representative LLMs on RGB to diagnose the challenges of current LLMs when applying RAG. Evaluation reveals that while LLMs exhibit a certain degree of noise robustness, they still struggle significantly in terms of negative rejection, information integration, and dealing with false information. The aforementioned assessment outcomes indicate that there is still a considerable journey ahead to effectively apply RAG to LLMs.

9. [9] Benchmarking Retrieval-Augmented Generation for Medicine

   Guangzhi Xiong, Qiao Jin, Zhiyong Lu, Aidong Zhang · 2024 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · cited 541 times · doi:10.18653/v1/2024.findings-acl.372

   Recent benchmark médical du RAG · method

   Why it is here: Le benchmark compare de nombreuses combinaisons de corpus, récupérateurs et modèles en médecine, montrant l’importance des choix d’évaluation.

   Abstract

   While large language models (LLMs) have achieved state-of-the-art performance on a wide range of medical question answering (QA) tasks, they still face challenges with hallucinations and outdated knowledge.Retrievalaugmented generation (RAG) is a promising solution and has been widely adopted.However, a RAG system can involve multiple flexible components, and there is a lack of best practices regarding the optimal RAG setting for various medical purposes.To systematically evaluate such systems, we propose the Medical Information Retrieval-Augmented Generation Evaluation (MIRAGE), a first-of-its-kind benchmark including 7,663 questions from five medical QA datasets.Using MIRAGE, we conducted large-scale experiments with over 1.8 trillion prompt tokens on 41 combinations of different corpora, retrievers, and backbone LLMs through the MEDRAG toolkit introduced in this work.Overall, MEDRAG improves the accuracy of six different LLMs by up to 18% over chain-of-thought prompting, elevating the performance of GPT-3.5 and Mixtral to GPT-4level.Our results show that the combination of various medical corpora and retrievers achieves the best performance.In addition, we discovered a log-linear scaling property and the "lostin-the-middle" effects in medical RAG.We believe our comprehensive evaluations can serve as practical guidelines for implementing RAG systems for medicine 1 .

10. [10] Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

    Wenhao Yu, Hongming Zhang, Xiaoman Pan, Peixin Cao et al. · 2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · cited 196 times · doi:10.18653/v1/2024.emnlp-main.813

    Recent Notes de lecture séquentielles · method

    Why it is here: Propose une méthode pour limiter les erreurs dues aux documents non pertinents et gérer l’incertitude en RAG.

    Abstract

    Retrieval-augmented language model (RALM) represents a significant advancement in mitigating factual hallucination by leveraging external knowledge sources.However, the reliability of the retrieved information is not always guaranteed, and the retrieval of irrelevant data can mislead the response generation.Moreover, standard RALMs frequently neglect their intrinsic knowledge due to the interference from retrieved information.In instances where the retrieved information is irrelevant, RALMs should ideally utilize their intrinsic knowledge or, in the absence of both intrinsic and retrieved knowledge, opt to respond with "unknown" to avoid hallucination.In this paper, we introduces CHAIN-OF-NOTE (CON), a novel approach to improve robustness of RALMs in facing noisy, irrelevant documents and in handling unknown scenarios.The core idea of CON is to generate sequential reading notes for each retrieved document, enabling a thorough evaluation of their relevance to the given question and integrating this information to formulate the final answer.Our experimental results show that GPT-4, when equipped with CON, outperforms the CHAIN-OF-THOUGHT approach.Besides, we utilized GPT-4 to create 10K CON data, subsequently trained on LLaMa-2 7B model.Our experiments across four open-domain QA benchmarks show that fine-tuned RALMs equipped with CON significantly outperform standard fine-tuned RALMs.

11. [11] Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases

    Jiarui Li, Ye Yuan, Zehua Zhang · 2024 · arXiv (Cornell University) preprint · cited 176 times · doi:10.48550/arxiv.2403.10446

    Recent Preprint RAG et fine-tuning spécialisé · experiment

    Why it is here: Évalue un système combinant RAG et fine-tuning pour des requêtes privées et temporelles, et discute les limites du fine-tuning seul.

    Abstract · conclusions of the full text read too

    We proposed an end-to-end system design towards utilizing Retrieval Augmented Generation (RAG) to improve the factual accuracy of Large Language Models (LLMs) for domain-specific and time-sensitive queries related to private knowledge-bases. Our system integrates RAG pipeline with upstream datasets processing and downstream performance evaluation. Addressing the challenge of LLM hallucinations, we finetune models with a curated dataset which originates from CMU's extensive resources and annotated with the teacher model. Our experiments demonstrate the system's effectiveness in generating more accurate answers to domain-specific and time-sensitive inquiries. The results also revealed the limitations of fine-tuning LLMs with small-scale and skewed datasets. This research highlights the potential of RAG systems in augmenting LLMs with external datasets for improved performance in knowledge-intensive tasks. Our code and models are available on Github.

12. [12] Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs

    Oded Ovadia, Menachem Brief, Moshik Mishaeli, Oren Elisha · 2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · cited 340 times · doi:10.18653/v1/2024.emnlp-main.15

    Recent Comparaison RAG et fine-tuning · experiment

    Why it is here: Compare directement le RAG et le fine-tuning non supervisé sur plusieurs tâches intensives en connaissances.

    Abstract

    Large language models (LLMs) encapsulate a vast amount of factual information within their pre-trained weights, as evidenced by their ability to answer diverse questions across different domains.However, this knowledge is inherently limited, relying heavily on the characteristics of the training data.Consequently, using external datasets to incorporate new information or refine the capabilities of LLMs on previously seen information poses a significant challenge.In this study, we compare two common approaches: unsupervised fine-tuning and retrieval-augmented generation (RAG).We evaluate both approaches on a variety of knowledge-intensive tasks across different topics.Our findings reveal that while unsupervised fine-tuning offers some improvement, RAG consistently outperforms it, both for existing knowledge encountered during training and entirely new knowledge.Moreover, we find that LLMs struggle to learn new factual information through unsupervised fine-tuning, and that exposing them to numerous variations of the same fact during training could alleviate this problem.

13. [13] Honest AI: Fine-Tuning "Small" Language Models to Say "I Don't Know", and Reducing Hallucination in RAG

    Xinxi Chen, Li Wang, Wei Wu, Qi Tang et al. · 2024 · arXiv (Cornell University) preprint · cited 20 times · doi:10.48550/arxiv.2410.09699

    Recent Preprint fine-tuning et variantes RAG · experiment

    Why it is here: Évalue conjointement le fine-tuning de petits modèles et plusieurs stratégies RAG, et indique que le RAG seul ne suffit pas toujours.

    Abstract · conclusions of the full text read too

    Hallucination is a key roadblock for applications of Large Language Models (LLMs), particularly for enterprise applications that are sensitive to information accuracy. To address this issue, two general approaches have been explored: Retrieval-Augmented Generation (RAG) to supply LLMs with updated information as context, and fine-tuning the LLMs with new information and desired output styles. In this paper, we propose Honest AI: a novel strategy to fine-tune "small" language models to say "I don't know" to reduce hallucination, along with several alternative RAG approaches. The solution ranked 1st in Task 2 for the false premise question. The alternative approaches include using RAG with search engine and knowledge graph results, fine-tuning base LLMs with new information and combinations of both approaches. Although all approaches improve the performance of the LLMs, RAG alone does not significantly improve the performance and fine-tuning is needed for better results. Finally, the hybrid approach achieved the highest score in the CRAG benchmark. In addition, our approach emphasizes the use of relatively small models with fewer than 10 billion parameters, promoting resource efficiency.

14. [14] RAG-Guided Large Language Models for Visual Spatial Description with Adaptive Hallucination Corrector

    Jun Yu, Yunxiang Zhang, Zerui Zhang, Yang Zhao et al. · 2024 · ACM International Conference on Multimedia (ACM MM) · cited 13 times · doi:10.1145/3664647.3688990

    Recent RAG avec correcteur adaptatif · experiment

    Why it is here: Évalue un RAG corrigé pour la description spatiale visuelle et rapporte moins d’hallucinations, sans comparaison au fine-tuning seul.

    Abstract

    Visual Spatial Description (VSD) is an emerging image-to-text task which aims at generating descriptions of the spatial relationships between given objects in an image. In this paper, we apply Retrieval-Augmented Generation (RAG) technology in guiding Multimodal Large Language Models (MLLMs) for the task of VSD, complemented by an Adaptive Hallucination Corrector, and further fine-tuning them to bolster semantic understanding and overall model efficacy. We found that our approach demonstrated higher accuracy and fewer hallucination errors in both spatial relationship classification and visual language description tasks within the VSD task, achieving state-of-the-art results.

15. [15] RAG-HAT: A Hallucination-Aware Tuning Pipeline for LLM in Retrieval-Augmented Generation

    Juntong Song, Xingguang Wang, Juno Zhu, Yuanhao Wu et al. · 2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · cited 47 times · doi:10.18653/v1/2024.emnlp-industry.113

    Recent ajustement sensible aux hallucinations · method

    Why it is here: Décrit une chaîne de détection et de correction des hallucinations en RAG, sans comparaison explicite au fine-tuning seul.

    Abstract

    Retrieval-augmented generation (RAG) has emerged as a significant advancement in the field of large language models (LLMs).By integrating up-to-date information not available during their initial training, RAG greatly enhances the practical utility of LLMs in realworld applications.However, even with RAG, LLMs can still produce inaccurate outputs, such as distorting or misinterpreting source content, posing risks in high-trust scenarios.To address these issues, we introduce a novel approach called Hallucination Aware Tuning (HAT).This method involves training hallucination detection models that generate detection labels and provide detailed descriptions of the detected hallucinations.Utilizing these detection results-particularly the hallucination descriptions-GPT-4 Turbo is employed to correct any detected hallucinations.The corrected outputs, free of hallucinations, along with the original versions, are used to create a preference dataset for Direct Preference Optimization (DPO) training.The fine-tuning through DPO leads to LLMs that exhibit a reduced rate of hallucinations and deliver improved answer quality.

16. [16] Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models

    Seiji Maekawa, Hayate Iso, Sairam Gurajada, Nikita Bhutani · 2024 · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · cited 24 times · doi:10.18653/v1/2024.naacl-long.308

    Recent Évaluation des effets de récupération · experiment

    Why it is here: Étudie quand la récupération aide ou nuit aux modèles de langage, éclairant les conditions d’évaluation, sans comparer au fine-tuning seul.

    Abstract

    Seiji Maekawa, Hayate Iso, Sairam Gurajada, Nikita Bhutani. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

17. [17] Exploring RAG Solutions to Reduce Hallucinations in LLMs

    Samar AboulEla, Paria Zabihitari, Nourhan Ibrahim, Majid Afshar et al. · 2025 · IEEE International Systems Conference (SysCon) · cited 29 times · doi:10.1109/syscon64521.2025.11014810

    Recent comparaison de systèmes RAG · experiment

    Why it is here: Compare RAG naïf et Graph RAG à des modèles de base et un modèle fine-tuné sur plusieurs jeux de données.

    Abstract

    Large Language Models (LLMs) often face challenges in generating accurate and reliable information, particularly in knowledge-intensive tasks. This limitation, referred to as hallucination, occurs when models produce content that is incorrect, irrelevant, or unsupported by evidence. Retrieval Augmented Generation (RAG) solutions provide a promising approach by integrating relevant external knowledge, enabling models to generate factually grounded responses. This study evaluates the performance of a base LLM model, a fine-tuned DistilBERT model, and two RAG architectures, Naïve RAG and Graph RAG, to study their impact on reducing hallucinations and enhancing contextual understanding. Using subsets of HaluEval, Squad-V2, and TriviaQA benchmark datasets, the base model achieved accuracies of 10.18%, 12.67%, and 5.46% respectively; Naive RAG resulted in 44.56%, 19.04%, and 35.32% accuracies; while the fine-tuned LLM model's accuracies were 72.5%, 72.31%, and 88.7% respectively. Graph RAG resulted in 8.85% and 15.12% accuracies using Squad-V2 and TriviaQA, respectively. Our findings show that while fine-tuned LLMs outperform baseline models, incorporating RAG solutions did not result in significant performance improvements, suggesting that the incorporation of external knowledge may not always align with the needs of the task. Experiments demonstrate that Graph RAG handles complex queries by leveraging relationships within structured knowledge graphs. Data organized as a knowledge graph may enable Graph RAG solutions reach their full potential by utilizing their capacity to efficiently retrieve contextually relevant information. Although computing complexity remains a restriction, this study shows that RAG topologies might not consistently enhance LLM reliability in practical situations.

18. [18] Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation

    Lee, Zhan Peng, Lin, Andre, Tan, Calvin · 2025 · arXiv (Cornell University) preprint · cited 11 times · doi:10.48550/arxiv.2505.10792

    Recent Preprint fine-tuning pour RAG imparfait · experiment

    Why it is here: Compare une méthode de fine-tuning conçue pour les erreurs de récupération à un modèle de base et rapporte un gain de précision factuelle.

    Abstract · conclusions of the full text read too

    Retrieval-Augmented Generation (RAG) has emerged as a powerful framework to improve factuality in large language models (LLMs) by grounding their outputs in retrieved documents. However, ensuring perfect retrieval of relevant information remains challenging, and when irrelevant content is passed downstream to an LLM, it can lead to hallucinations. In this work, we propose Finetune-RAG, a simple and effective fine-tuning approach that features the first-of-its-kind RAG training dataset constructed to mimic real-world imperfections. Experimental results show that Finetune-RAG improves factual accuracy by 21.2% over the base model. We also propose Bench-RAG, an LLM-as-a-judge evaluation pipeline that stress tests models under realistic imperfect retrieval scenarios. Our codebase and dataset are fully open sourced for community use.

19. [19] Hallucination Mitigation for Retrieval-Augmented Large Language Models: A Review

    Wan Zhang, Jing Zhang · 2025 · Mathematics · cited 195 times · doi:10.3390/math13050856

    Recent Review revue des hallucinations en RAG · review

    Why it is here: Passe en revue les causes et mesures d’atténuation propres au RAG, sans comparaison directe avec le fine-tuning seul.

    Abstract

    Retrieval-augmented generation (RAG) leverages the strengths of information retrieval and generative models to enhance the handling of real-time and domain-specific knowledge. Despite its advantages, limitations within RAG components may cause hallucinations, or more precisely termed confabulations in generated outputs, driving extensive research to address these limitations and mitigate hallucinations. This review focuses on hallucination in retrieval-augmented large language models (LLMs). We first examine the causes of hallucinations from different sub-tasks in the retrieval and generation phases. Then, we provide a comprehensive overview of corresponding hallucination mitigation techniques, offering a targeted and complete framework for addressing hallucinations in retrieval-augmented LLMs. We also investigate methods to reduce the impact of hallucination through detection and correction. Finally, we discuss promising future research directions for mitigating hallucinations in retrieval-augmented LLMs.

20. [20] Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines

    Siru Liu, Allison B. McCoy, Adam T. Wright · 2025 · Journal of the American Medical Informatics Association · cited 276 times · doi:10.1093/jamia/ocaf008

    Recent Review Méta-analyse du RAG biomédical · review

    Why it is here: Synthétise et quantifie les comparaisons empiriques entre performances LLM de base et RAG en biomédecine.

    Abstract · conclusions of the full text read too

    OBJECTIVE: The objectives of this study are to synthesize findings from recent research of retrieval-augmented generation (RAG) and large language models (LLMs) in biomedicine and provide clinical development guidelines to improve effectiveness. MATERIALS AND METHODS: We conducted a systematic literature review and a meta-analysis. The report was created in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 analysis. Searches were performed in 3 databases (PubMed, Embase, PsycINFO) using terms related to "retrieval augmented generation" and "large language model," for articles published in 2023 and 2024. We selected studies that compared baseline LLM performance with RAG performance. We developed a random-effect meta-analysis model, using odds ratio as the effect size. RESULTS: Among 335 studies, 20 were included in this literature review. The pooled effect size was 1.35, with a 95% confidence interval of 1.19-1.53, indicating a statistically significant effect (P = .001). We reported clinical tasks, baseline LLMs, retrieval sources and strategies, as well as evaluation methods. DISCUSSION: Building on our literature review, we developed Guidelines for Unified Implementation and Development of Enhanced LLM Applications with RAG in Clinical Settings to inform clinical applications using RAG. CONCLUSION: Overall, RAG implementation showed a 1.35 odds ratio increase in performance compared to baseline LLMs. Future research should focus on (1) system-level enhancement: the combination of RAG and agent, (2) knowledge-level enhancement: deep integration of knowledge into LLM, and (3) integration-level enhancement: integrating RAG systems within electronic health records.

21. [21] Retrieval-Augmented Generation and Hallucination in Large Language Models: A Scholarly Overview

    Sagar Gupta · 2025 · Scholars Journal of Engineering and Technology · cited 13 times · doi:10.36347/sjet.2025.v13i05.003

    Recent Review revue de RAG anti-hallucination · review

    Why it is here: Présente les mécanismes et l’efficacité du RAG, mais l’abstract ne compare pas directement avec le fine-tuning.

    Abstract · conclusions of the full text read too

    Large Language Models (LLMs) have revolutionized natural language processing tasks, yet they often suffer from "hallucination” the confident generation of factually incorrect information. Retrieval-Augmented Generation (RAG) has emerged as a promising technique to mitigate hallucinations by grounding model responses in external documents. This article explores the underlying causes of hallucinations in LLMs, the mechanisms and architectures of RAG systems, their effectiveness in reducing hallucinations, and ongoing challenges. We conclude with a discussion of future directions for integrating retrieval mechanisms more seamlessly into generative architecture.

22. [22] A Systematic Review of Retrieval-Augmented Generation Techniques for Mitigating Hallucinations in Chatbot Systems

    Shahirah Mohamed Hatim, Afif Amsyar Khairul Nizam, Nurhawa Shamsudin, Nur Azmina Mohamad Zamani et al. · 2026 · International Conference on Artificial Intelligence and Data Sciences (AiDAS) · cited 0 times · doi:10.1109/aidas71914.2026.11712729

    Recent Review revue systématique du RAG · review

    Why it is here: Synthétise architectures, récupération et évaluations du RAG en chatbots, mais l’abstract ne rapporte pas de comparaison au fine-tuning seul.

    Abstract

    Large language model (LLM) chatbots are increasingly deployed in education, health care, and decision-support settings, yet their reliability is limited by hallucination: fluent responses that are factually incorrect, weakly grounded, or inconsistent with supplied evidence. Retrieval-Augmented Generation (RAG) addresses this limitation by conditioning generation on external knowledge retrieved at inference time. This paper presents a systematic review of RAG techniques for hallucination mitigation in chatbot systems. Following the PRISMA 2020 process, studies published from 2020 to 2026 were searched in Scopus, IEEE Xplore, and Google Scholar. After duplicate removal, screening, and full-text eligibility assessment, 23 studies were synthesized qualitatively. The review analyzes RAG architectures, retrieval strategies, evidence-integration mechanisms, and evaluation practices. Findings show that advanced and modular RAG architectures generally improve factuality and faithfulness over purely parametric LLMs, especially when retrieval is paired with re-ranking, evidence filtering, verification, or citation-aware generation. However, mitigation remains constrained by retrieval noise, latency, limited benchmark standardization, and weak integration between hallucination detection and correction. The paper concludes with research directions for trustworthy, efficient, and explainable RAG-based chatbot systems.

23. [23] Hallucination in Large Language Models and Retrieval-Augmented Generation: Mechanisms, Mitigation, and Evaluation

    Haopeng Yang · 2026 · Theoretical and Natural Science · cited 0 times · doi:10.54254/2753-8818/2026.dl34055

    Recent Review revue des mécanismes et évaluations · review

    Why it is here: Analyse les causes, atténuations et évaluations des hallucinations en RAG, utiles pour interpréter les différences selon les réglages.

    Abstract · conclusions of the full text read too

    Large language models have demonstrated strong generative capability in question answering, dialogue, and other knowledge-intensive tasks. However, their outputs remain vulnerable to hallucination, including factual errors, unsupported claims, spurious citations, and distorted reasoning. Retrieval-augmented generation (RAG) has been proposed as a practical remedy because it supplements parametric knowledge with external evidence retrieved at inference time. Yet RAG does not guarantee truthfulness or attribution by default. Errors may arise during query formulation, document retrieval, evidence aggregation, and answer grounding. This paper reviews the relationship between hallucination and RAG from three perspectives: mechanism, mitigation, and evaluation. First, the major causes of hallucination in both vanilla LLMs and RAG-enhanced systems are analyzed. Second, the principal mitigation strategies are organized into retrieval optimization, evidence-grounded generation, and post-generation verification. Third, the main evaluation dimensions are examined, including factuality, faithfulness, attribution quality, and retrieval relevance. It is argued that RAG should not be treated as a complete solution to hallucination. Its value lies in enabling externally grounded generation, but its effectiveness depends on the reliability of retrieval, the fidelity of evidence use, and the rigor of evaluation. Future work should prioritize attribution-aware generation, conflict-sensitive reasoning, and unified evaluation protocols for trustworthy LLM systems.

24. [24] Integrating Fine-Tuning and Retrieval-Augmented Generation for Healthcare AI Systems: A Scoping Review

    Bernardo Gabriele Collaço, Prabha Srinivasagam, Cesar Abraham Gomez-Cabello, Syed Ali Haider et al. · 2026 · Bioengineering · cited 8 times · doi:10.3390/bioengineering13020225

    Recent Review Revue des hybrides santé · review

    Why it is here: Synthétise les études empiriques combinant fine-tuning et RAG en santé, directement pertinentes pour leurs effets complémentaires.

    Abstract · conclusions of the full text read too

    (1) Background: Large language models (LLMs) show promise in healthcare but are constrained by hallucinations, static knowledge, and limited domain specificity. Fine-tuning (FT) and retrieval-augmented generation (RAG) offer complementary solutions, with FT embedding domain reasoning and RAG enabling dynamic, up-to-date knowledge access. Hybrid FT + RAG frameworks have been proposed to improve factual accuracy and clinical reliability. This scoping review synthesizes current evidence on such hybrids in healthcare AI. (2) Methods: The search across PubMed, IEEE Xplore, Google Scholar, and Embase identified studies implementing explicit FT + RAG hybrids in healthcare or biomedical tasks. Eligible studies reported empirical evaluations of LLM performance or behavior. Data were extracted on base models, FT strategies, RAG architectures, applications, and performance outcomes. (3) Results: Seven studies met inclusion criteria. FT + RAG systems consistently outperformed FT-only or RAG-only approaches across QA, clinical summarization, report generation, and decision support tasks. Parameter-efficient FT methods (e.g., LoRA) were common, while RAG implementations varied (dense, hybrid, hierarchical, multimodal, federated). Reported benefits included improved accuracy, reduced hallucination, and greater clinician preference and feasibility in protected settings. (4) Conclusions: FT + RAG frameworks represent a promising direction for clinically grounded healthcare AI, combining domain-specific reasoning with transparent, up-to-date retrieval. Future work should prioritize standardized evaluation, workflow integration, and governance to enable safe deployment.

25. [25] Mitigating Hallucination on Hallucination in RAG via Ensemble Voting

    Zequn Xie, Zhengyang Sun · 2026 · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · cited 3 times · doi:10.1109/cscwd68734.2026.11582530

    Recent vote multi-agents pour RAG · experiment

    Why it is here: Compare empiriquement une méthode de RAG aux hallucinations induites par de mauvaises récupérations, sans référence au fine-tuning seul.

    Abstract · conclusions of the full text read too

    Retrieval-Augmented Generation (RAG) aims to reduce hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, RAG introduces a critical challenge: hallucination on hallucination," where flawed retrieval results mislead the generation model, leading to compounded hallucinations. To address this issue, we propose VOTE-RAG, a novel, training-free framework with a two-stage structure and efficient, parallelizable voting mechanisms. VOTE-RAG includes: (1) Retrieval Voting, where multiple agents generate diverse queries in parallel and aggregate all retrieved documents; (2) Response Voting, where multiple agents independently generate answers based on the aggregated documents, with the final output determined by majority vote. We conduct comparative experiments on six benchmark datasets. Our results show that VOTE-RAG achieves performance comparable to or surpassing more complex frameworks. Additionally, VOTE-RAG features a simpler architecture, is fully parallelizable, and avoids the problem drift" risk. Our work demonstrates that simple, reliable ensemble voting is a superior and more efficient method for mitigating RAG hallucinations.

26. [26] RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

    Fabian Ridder, Laurin Lessel, Malte Schilling · 2026 · International Joint Conference on Neural Networks (IJCNN) · cited 1 time · doi:10.1109/ijcnn69854.2026.11728700

    Recent fine-tuning avec détection intégrée · method

    Why it is here: Propose un fine-tuning sensible aux hallucinations en contexte RAG, utile pour comprendre l’interaction des approches, mais sans comparaison directe annoncée.

    Abstract · conclusions of the full text read too

    Retrieval-Augmented Generation (RAG) is widely used to augment the input to Large Language Models (LLMs) with external information, such as recent or domain-specific knowledge. Nonetheless, current models still produce closed-domain hallucinations and generate content that is unsupported by the retrieved context. Current detection approaches typically treat hallucination as a post-hoc problem, relying on black-box consistency checks or probes over frozen internal representations. In this work, we demonstrate that hallucination detection based on internal state representation can also serve as a direct training signal. We introduce RAGognize, a dataset of naturally occurring closed-domain hallucinations with token-level annotations, and RAGognizer, a hallucination-aware fine-tuning approach that integrates a lightweight detection head into an LLM, allowing for the joint optimization of language modeling and hallucination detection. This joint objective forces the model to improve the separability of its internal states regarding hallucinations while simultaneously learning to generate well-formed and meaningful responses. Across multiple benchmarks, RAGognizer achieves state-of-the-art token-level hallucination detection while substantially reducing hallucination rates during generation, without degrading language quality or relevance.

27. [27] Reducing Hallucinations in Multi-Hop Question Answering Using Wikidata-Based Retrieval-Augmented Generation

    Razi Alsayyed, Shukri Khelfa, Amjad Hawash · 2026 · ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS) · cited 0 times · doi:10.1109/icetsis68266.2026.11549183

    Recent RAG structuré avec Wikidata · experiment

    Why it is here: Évalue empiriquement le RAG sur des questions multi-sauts avec plusieurs mesures d’hallucination, mais sans comparaison au fine-tuning.

    Abstract

    Hallucinations continue to significantly restrict large language models (LLMs), particularly in fact-finding and knowledge-intensive tasks. Retrieval-Augmented Generation (RAG), which bases model outputs on external knowledge sources, addresses this problem. This study examines the impact of integrating structured knowledge from Wikidata into an LLM-based question-answering pipeline to reduce hallucinated material. The proposed Wikidata-based RAG technique injects this structured context into the LLM prompt before response synthesis by first extracting entities from the input question and then gathering candidate entities and verified attributes from Wikidata. Both the prompt-only baseline models and the Wikidata-grounded RAG system were used to evaluate 80 multi-hop compositional problems. The proposed RAG model is evaluated using four hallucination evaluation methods. The model is evaluated against three prompt-only baselines: the AimonLabs hallucination-detection model, the Vectara HHEM, and an LLM-as-a-Judge rubric. Retrieved-Context Faithfulness (RAGTruth-style evaluation) is assessed using the last evaluation method. (Vectara HHEM: 77.5% vs 38.75% factual replies; AIMON: 60% vs 32.5% factual responses; LLM-as-a-Judge: 75% vs 38.75% factual responses) RAG consistently surpasses the baseline across all four evaluators. 54 of 79 assessed examples had a retrieved-context fidelity of 68.35%. This suggests that the observed frequency of hallucinations is influenced by the evaluation approach used. These results show how fact accuracy may be greatly increased by basing LLMs on structured knowledge from Wikidata. They also show how important it is to have several assessors when studying hallucinations.

28. [28] Retrieval-Augmented Generation (RAG) vs. Fine-Tuning: A Systematic Review of Hallucination Mitigation in Medical LLMs

    Hassan Jubair · 2026 · cited 1 time · doi:10.22541/au.176903295.54191070/v1

    Recent Review Preprint revue comparative en médecine · review

    Why it is here: Compare explicitement RAG et fine-tuning dans des études médicales et rapporte leur efficacité relative contre les hallucinations.

    Abstract

    Background: The integration of Large Language Models (LLMs) into healthcare is currently hindered by the critical risk of "hallucinations", the generation of plausible but factually incorrect medical information.Two primary adaptation strategies have emerged to address this: Fine-Tuning (FT), which updates model parameters to internalize domain knowledge, and Retrieval-Augmented Generation (RAG), which grounds model outputs in external evidence at inference time.Objective: This systematic review synthesizes findings from 50 peer-reviewed studies published between 2023 and 2025 to evaluate the comparative effectiveness of RAG versus Fine-Tuning in medical contexts.Results: Evidence consistently demonstrates that RAG is the superior strategy for clinical safety, significantly reducing hallucination rates by dynamically accessing up-to-date knowledge without the need for retraining.Conversely, while Fine-Tuning excels at adapting to domain-specific syntax and clinical reasoning styles, it incurs high computational costs and carries a significant risk of "catastrophic forgetting," where general knowledge is degraded.Conclusion: We conclude that while Fine-Tuning remains valuable for specialization, RAG provides the necessary factual guardrails for patient-facing applications.Future development should prioritize "Hybrid" frameworks that synergize the reasoning depth of Fine-Tuning with the factual integrity of RAG.

29. [29] Retrieval-Augmented Generation for Hallucination Mitigation in Large Language Models: Basic RAG, Self-RAG, and Corrective RAG

    Xiaojie Yang · 2026 · ITM Web of Conferences · cited 0 times · doi:10.1051/itmconf/20269301009

    Recent Review comparaison de méthodes RAG · review

    Why it is here: Examine plusieurs variantes de RAG anti-hallucination, mais pas leur comparaison avec le fine-tuning seul.

    Abstract

    Although large language models have shown strong ability in generating text, they can still give answers that look fluent but are factually incorrect or lack clear support, especially when the questions are related to factual knowledge, specific fields, or information that changes quickly. This issue is commonly called hallucination. Retrieval-Augmented Generation (RAG), provides a useful way to reduce this problem, because it lets the model use external documents before producing the final answer. This paper mainly discusses three representative RAG-based methods for reducing hallucinations, including Basic RAG, Self-RAG, and Corrective RAG. Basic RAG uses a simple process of retrieving first and then generating, so the generated content can be more closely connected with factual information. Self-RAG adds a self-reflection process to this basic structure, and the model can decide whether retrieval is necessary, as well as whether the answer has enough evidence support. Corrective RAG pays more attention to the quality of retrieved information; it checks and corrects poor retrieved content before the generation stage. According to their working processes, limitations, and related experimental results, this paper also compares these three methods from several aspects, such as factual reliability, retrieval control, use of evidence, system complexity, and suitable application scenarios.

30. [30] Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generation

    Qianchi Zhang, Hainan Zhang, Liang Pang, Hong-Wei Zheng et al. · 2026 · Proceedings of the conference - Association for Computational Linguistics. Meeting · cited 37 times · doi:10.18653/v1/2026.acl-long.1188

    Recent robustesse aux permutations documentaires · experiment

    Why it is here: Montre que l’ordre des documents récupérés influe sur les hallucinations, un facteur important des résultats RAG selon les réglages.

    Abstract

    Retrieval-Augmented Generation (RAG) has become a key paradigm for reducing factual hallucinations in Large Language Models (LLMs), yet little is known about how the order of retrieved documents affects model behavior.We empirically show that under a Top-5 retrieval setting with the gold document included, LLM answers vary substantially across permutations of the retrieved set, even when the gold document is fixed in the first position.This reveals a previously underexplored sensitivity to retrieval permutations.Although existing robust RAG methods focus primarily on enhancing LLM robustness to low-quality retrieval and mitigating positional bias to distribute attention fairly over long contexts, neither approach directly addresses permutation sensitivity.In this paper, we propose Stable-RAG, which exploits permutation sensitivity estimation to mitigate permutation-induced hallucinations.Stable-RAG runs the generator under multiple retrieval orders, clusters hidden states, and decodes from a cluster-center representation that captures the dominant reasoning pattern.It then uses these reasoning results to align hallucinated outputs toward the correct answer, encouraging the model to produce consistent and accurate predictions across document permutations.Experiments on three QA datasets show that Stable-RAG improves answer accuracy, reasoning consistency, and generalization across datasets, retrievers, and input lengths compared with strong baselines 1 .