Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Très proche Pas pertinent

   Retrieval-augmented Generation across Heterogeneous Knowledge

   W. Yu · 2022 · North American Chapter of the Association for Computational Linguistics · cité 73 fois · Pas encore résumé

   Résumé scientifique

   Retrieval-augmented generation (RAG) methods have been receiving increasing attention from the NLP community and achieved state-of-the-art performance on many NLP downstream tasks. Compared with conventional pre-trained generation models, RAG methods have remarkable advantages such as easy knowledge acquisition, strong scalability, and low training cost. Although existing RAG models have been applied to various knowledge-intensive NLP tasks, such as open-domain QA and dialogue systems, most of the work has focused on retrieving unstructured text documents from Wikipedia. In this paper, I first elaborate on the current obstacles to retrieving knowledge from a single-source homogeneous corpus.…

2. Très proche Pas pertinent

   Dynamic and Parametric Retrieval-Augmented Generation

   Wei-Hang Su, Qing-Yao Ai, Jingtao Zhan et al. · 2025 · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · cité 16 fois · Pas encore résumé

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) has become a foundational paradigm for enhancing large language models (LLMs) with external knowledge, playing an important role in modern information retrieval and knowledge-intensive NLP applications. Standard RAG systems typically adopt a static retrieve-then-generate pipeline and rely on in-context knowledge injection, which can be suboptimal for complex tasks that require multihop reasoning, adaptive information access, and deeper integration of external knowledge. Motivated by these limitations, the research community has moved beyond static retrieval and in-context knowledge injection. Among the emerging directions, this tutorial delves into two ra…

3. Très proche Pas pertinent

   Retrieval-augmented generation for natural language processing: a survey

   Shang-Yu Wu, Ying Xiong, Yu-Fei Cui et al. · 2024 · Artificial Intelligence Review · cité 185 fois · Pas encore résumé

   Résumé scientifique

   Large language models (LLMs) have achieved strong empirical performance in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge base to augment LLMs, mitigates these limitations. This paper presents a systematic review of RAG techniques for natural language processing (NLP), with a focus on retrievers and retrieval fusions. We introduce a novel taxonomy of retrieval fusions, such as query-based, logits-based, latent, and …

4. Très proche Pas pertinent

   Retrieval-Augmented Generation for Large Language Models: A Survey

   Yunfan Gao, Yun Xiong, Xinyu Gao et al. · 2023 · Computation and Language · preprint, non relu par les pairs · cité 4 427 fois · Résumé IA disponible

   Résumé scientifique

   Large Language Models (LLMs) demonstrate significant capabilities but face challenges such as hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes. Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases. This enhances the accuracy and credibility of the models, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domain-specific information. RAG synergistically merges LLMs' intrinsic knowledge with the vast, dynamic repositories of external databases. This comprehensive review paper offers a detailed examination of the progression of RA…

5. Très proche Pas pertinent

   Enhancing Retrieval-Augmented Generation: A Study of Best Practices

   Siran Li, Linus Stenzel, Carsten Eickhoff et al. · 2025 · Computation and Language · cité 65 fois · Résumé IA disponible

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) systems have recently shown remarkable advancements by integrating retrieval mechanisms into language models, enhancing their ability to produce more accurate and contextually relevant responses. However, the influence of various components and configurations within RAG systems remains underexplored. A comprehensive understanding of these elements is essential for tailoring RAG systems to complex retrieval tasks and ensuring optimal performance across diverse applications. In this paper, we develop several advanced RAG system designs that incorporate query expansion, various novel retrieval strategies, and a novel Contrastive In-Context Learning RAG. Our …

6. Très proche Pas pertinent

   SegMem-RAG: Adaptive Memory for Retrieval-Augmented Generation in Open-Ended Knowledge Environments

   Xuanbo Fan, Tian-Qi Zhao, Yi Cheng et al. · 2026 · AAAI Conference on Artificial Intelligence · cité 2 fois · Pas encore résumé

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) improves the factual accuracy of large language models by grounding responses in external content. However, most RAG systems assume access to static and well-organized corpora with fixed retrieval logic. In practice, real-world sources are heterogeneous and unlabeled, including user-uploaded documents, manuals, and datasets. Effective access in such settings requires adaptive and self-directed retrieval behavior. We present SegMem‑RAG, a memory-augmented RAG framework that learns to route queries across multiple unlabeled corpora based on experience. It incrementally updates a structured memory and uses self-reflection to guide retrieval over time without…

7. Très proche Pas pertinent

   FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAG

   Xinping Zhao, Yan Zhong, Zetian Sun et al. · 2024 · Information Retrieval · cité 26 fois · Résumé IA disponible

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) prevails in Large Language Models. It mainly consists of retrieval and generation. The retrieval modules (a.k.a. retrievers) aim to find useful information used to facilitate generation modules (a.k.a. generators). As such, generators' performance largely depends on the effectiveness and efficiency of retrievers. However, the retrieval paradigm that we design and use remains flat, which treats the retrieval procedures as a one-off deal with constant granularity. Despite effectiveness, we argue that they suffer from two limitations: (1) flat retrieval exerts a significant burden on one retriever; (2) constant granularity limits the ceiling of retrieval per…

8. Très proche Pas pertinent

   MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge Discovery

   Hongjin Qian, Peitian Zhang, Zheng Liu et al. · 2024 · Computation and Language · cité 172 fois · Résumé IA disponible

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) leverages retrieval tools to access external databases, thereby enhancing the generation quality of large language models (LLMs) through optimized context. However, the existing retrieval methods are constrained inherently, as they can only perform relevance matching between explicitly stated queries and well-formed knowledge, but unable to handle tasks involving ambiguous information needs or unstructured knowledge. Consequently, existing RAG systems are primarily effective for straightforward question-answering tasks. In this work, we propose MemoRAG, a novel retrieval-augmented generation paradigm empowered by long-term memory. MemoRAG adopts a dual-sy…

Les travaux antérieurs sur lesquels il s’appuie

1. Proche Utilisé en profondeur par cet article Une méthode qu’il utilise Pas pertinent

   REALM: Retrieval-Augmented Language Model Pre-Training

   Kelvin Guu, Kenton Lee, Zora Tung et al. · 2020 · International Conference on Machine Learning · cité 3 608 fois · Pas encore résumé

   Résumé scientifique

   Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering. However, this knowledge is stored implicitly in the parameters of a neural network, requiring ever-larger networks to cover more facts. To capture knowledge in a more modular and interpretable way, we augment language model pre-training with a latent knowledge retriever, which allows the model to retrieve and attend over documents from a large corpus such as Wikipedia, used during pre-training, fine-tuning and inference. For the first time, we show how to pre-train such a knowledge retriever in an unsupervised manner, using masked language modeling a…

2. Proche Utilisé en profondeur par cet article Une méthode qu’il utilise Pas pertinent

   How Much Knowledge Can You Pack into the Parameters of a Language Model?

   Adam Roberts, Colin A. Raffel, Noam Shazeer · 2020 · Conference on Empirical Methods in Natural Language Processing · cité 1 157 fois · Pas encore résumé

   Résumé scientifique

   It has recently been observed that neural language models trained on unstructured text can implicitly store and retrieve knowledge using natural language queries. In this short paper, we measure the practical utility of this approach by fine-tuning pre-trained models to answer questions without access to any external context or knowledge. We show that this approach scales surprisingly well with model size and outperforms models that explicitly look up knowledge on the open-domain variants of Natural Questions and WebQuestions. To facilitate reproducibility and future work, we release our code and trained models.

3. Apparenté Utilisé en profondeur par cet article Une méthode qu’il utilise Pas pertinent

   Latent Retrieval for Weakly Supervised Open Domain Question Answering

   Kenton Lee, Ming-Wei Chang, K. Toutanova · 2019 · Annual Meeting of the Association for Computational Linguistics · cité 1 264 fois · Pas encore résumé

   Résumé scientifique

   Recent work on open domain question answering (QA) assumes strong supervision of the supporting evidence and/or assumes a blackbox information retrieval (IR) system to retrieve evidence candidates. We argue that both are suboptimal, since gold evidence is not always available, and QA is fundamentally different from IR. We show for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system. In this setting, evidence retrieval from all of Wikipedia is treated as a latent variable. Since this is impractical to learn from scratch, we pre-train the retriever with an Inverse Cloze Task. We evaluate on open versions of fi…

4. Proche Pas pertinent

   Retrieve and Refine: Improved Sequence Generation Models For Dialogue

   J. Weston, Emily Dinan, Alexander H. Miller · 2018 · SCAI@EMNLP · cité 212 fois · Pas encore résumé

   Résumé scientifique

   Sequence generation models for dialogue are known to have several problems: they tend to produce short, generic sentences that are uninformative and unengaging. Retrieval models on the other hand can surface interesting responses, but are restricted to the given retrieval set leading to erroneous replies that cannot be tuned to the specific context. In this work we develop a model that combines the two approaches to avoid both their deficiencies: first retrieve a response and then refine it – the final sequence generator treating the retrieval as additional context. We show on the recent ConvAI2 challenge task our approach produces responses superior to both standard retrieval and generation…

5. Apparenté Cite cet article Utilisé en profondeur par cet article Une méthode qu’il utilise Pas pertinent

   Dense Passage Retrieval for Open-Domain Question Answering

   Vladimir Karpukhin, Barlas Oğuz, Sewon Min et al. · 2020 · Conference on Empirical Methods in Natural Language Processing · cité 7 738 fois · Résumé IA disponible

   Résumé scientifique

   Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method. In this work, we show that retrieval can be practically implemented using dense representations alone, where embeddings are learned from a small number of questions and passages by a simple dual-encoder framework. When evaluated on a wide range of open-domain QA datasets, our dense retriever outperforms a strong Lucene-BM25 system largely by 9%-19% absolute in terms of top-20 passage retrieval accuracy, and helps our end-to-end QA system establish new state-of-the-art on multiple open-domain QA benchm…

Les articles qui le citent

1. Proche Pas pertinent

   Active Retrieval Augmented Generation

   Zhengbao Jiang, Frank F. Xu, Luyu Gao et al. · 2023 · Computation and Language · cité 675 fois · Résumé IA disponible

   Résumé scientifique

   Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval-augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout the generation process is essential. There have been some past efforts to retrieve information multiple times while generating outputs, which mostly re…

2. Proche Pas pertinent

   Benchmarking Large Language Models in Retrieval-Augmented Generation

   Jiawei Chen, Hongyu Lin, Xianpei Han et al. · 2023 · Computation and Language · cité 546 fois · Résumé IA disponible

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) is a promising approach for mitigating the hallucination of large language models (LLMs). However, existing research lacks rigorous evaluation of the impact of retrieval-augmented generation on different large language models, which make it challenging to identify the potential bottlenecks in the capabilities of RAG for different LLMs. In this paper, we systematically investigate the impact of Retrieval-Augmented Generation on large language models. We analyze the performance of different large language models in 4 fundamental abilities required for RAG, including noise robustness, negative rejection, information integration, and counterfactual robustness…

3. Très proche Pas pertinent

   From Lookups to Lessons: Turning Retrieved Knowledge into Parametric Memory

   Chang-Jiang Han, Bo-Wei He, Ye Yuan et al. · 2026 · Pas encore résumé

   Résumé scientifique

   Retrieval-augmented generation (RAG) helps large language models (LLMs) answer factual questions beyond their training data, but it adds storage and inference costs. Distilling previously retrieved information into parametric memories enables reuse without online retrieval. However, training the memory to mimic a retriever rewards accurate independent predictions, which do not necessarily improve the final output quality when combined with the LLM's existing predictions. To address this, we introduce ResMem, in which the LLM reads retrieved text offline to generate teacher targets. This design avoids storing massive token representations, unlike existing parametric memory approaches such as …

4. Proche Utilise cet article en profondeur Pas pertinent

   MAPS: Macro-Adaptive Parallel Reasoning for Regulating Search Momentum in Retrieval-Augmented Generation

   Xin-Ze Li, Ding-Wen Zhang, Zheng-Hao Liu et al. · 2026 · Pas encore résumé

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) improves the performance of Large Language Models (LLMs) by retrieving external knowledge. Recently, some RAG methods interleave reasoning and retrieval, enabling LLMs to progressively generate sub-queries and retrieve external knowledge. However, these methods may suffer from redundant or ineffective sub-queries and insufficiently diverse query exploration. Moreover, due to the tightly interleaved nature of reasoning and retrieval, it remains unclear how the evolving reasoning process influences query intent. To characterize the evolution of query intent during reasoning, we introduce search momentum, which captures the accumulated retrieval tendency ind…

5. Proche Pas pertinent

   Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

   Gautier Izacard, Édouard Grave · 2021 · Conference of the European Chapter of the Association for Computational Linguistics · cité 1 276 fois · Résumé IA disponible

   Résumé scientifique

   Generative models for open domain question answering have proven to be competitive, without resorting to external knowledge. While promising, this approach requires to use models with billions of parameters, which are expensive to train and query. In this paper, we investigate how much these models can benefit from retrieving text passages, potentially containing evidence. We obtain state-of-the-art results on the Natural Questions and TriviaQA open benchmarks. Interestingly, we observe that the performance of this method significantly improves when increasing the number of retrieved passages. This is evidence that generative models are good at aggregating and combining evidence from multipl…