Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Builds on this paper Cites this paper Not relevant

   KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models

   Zhen Zhang, Xinyu Wang, Yong Jiang et al. · 2024 · Conference on Empirical Methods in Natural Language Processing · cited 1 time · Not summarized yet

   KBM uses a knowledge-boundary model to decide when retrieval is needed, extending the paper’s study of boundary awareness under retrieval augmentation.

   Abstract

   Large Language Models (LLMs) often struggle with dynamically changing knowledge and handling unknown static information. Retrieval-Augmented Generation (RAG) is employed to tackle these challenges and has a significant impact on improving LLM performance. In fact, we find that not all questions need to trigger RAG. By retrieving parts of knowledge unknown to the LLM and allowing the LLM to answer the rest, we can effectively reduce both time and computational costs. In our work, we propose a Knowledge Boundary Model (KBM) to express the known/unknown of a given question, and to determine whether a RAG needs to be triggered. Experiments conducted on 11 English and Chinese datasets illustrate …

2. Very close Another approach to the problem Not relevant

   Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

   Baolin Peng, Michel Galley, Pengcheng He et al. · 2023 · Computation and Language · preprint, not peer reviewed · cited 562 times · AI summary available

   LLM-Augmenter grounds and revises responses using external knowledge and automated feedback, rather than studying models’ awareness of their factual knowledge boundaries.

   Abstract

   Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e.g., task-oriented dialog and question answering. However, applying LLMs to real-world, mission-critical applications remains challenging mainly due to their tendency to generate hallucinations and inability to use external knowledge.This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules. Our system makes the LLM generate responses grounded in consolidated external knowledge, e.g., stored in task-specific databases. It also iteratively revises LLM prompts to improve model responses using feedback generated by util…

3. Very close Another approach to the problem Not relevant

   Self-Knowledge Guided Retrieval Augmentation for Large Language Models

   Yi-Le Wang, Peng Li, Mao-Song Sun et al. · 2023 · Conference on Empirical Methods in Natural Language Processing · cited 114 times · Not summarized yet

   Self-Knowledge Guided Retrieval Augmentation uses models’ recognition of what they know to manage retrieved knowledge, addressing retrieval’s effects on answers.

   Abstract

   Large language models (LLMs) have shown superior performance without task-specific fine-tuning. Despite the success, the knowledge stored in the parameters of LLMs could still be incomplete and difficult to update due to the computational costs. As complementary, retrieval-based methods can offer non-parametric world knowledge and improve the performance on tasks such as question answering. However, we find that the retrieved knowledge does not always help and even has a negative impact on original responses occasionally. To better make use of both internal knowledge and external world knowledge, we investigate eliciting the model's ability to recognize what they know and do not know (which …

4. Very close Builds on this paper Cites this paper Not relevant

   Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Reranking

   Haoxiang Jin, Rong-Han Li, Zixiang Lu et al. · 2025 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   This work uses parametric knowledge and post-retrieval confidence to filter evidence and guide retrieval, extending inquiry into knowledge boundaries during retrieval.

   Abstract

   Large Language Models (LLMs) often generate inaccurate responses (hallucinations) when faced with questions beyond their knowledge scope. Retrieval-Augmented Generation (RAG) addresses this by leveraging external knowledge, but a critical challenge remains: determining whether retrieved contexts effectively enhance the model`s ability to answer specific queries. This challenge underscores the importance of knowledge boundary awareness, which current methods-relying on discrete labels or limited signals-fail to address adequately, as they overlook the rich information in LLMs` continuous internal hidden states. To tackle this, we propose a novel post-retrieval knowledge filtering approach. Fi…

5. Very close Builds on this paper Cites this paper Uses this paper in depth Not relevant

   Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering

   Zhihua Wen, Zhi-Liang Tian, Ze-Xin Jian et al. · 2024 · Neural Information Processing Systems · cited 29 times · Not summarized yet

   This work examines knowledge-boundary perception for partially answerable questions, extending the paper’s boundary analysis beyond open-domain questions with concrete answers.

   Abstract

   Large Language Models (LLMs) are widely used for knowledge-seeking yet suffer from hallucinations. The knowledge boundary (KB) of an LLM limits its factual understanding, beyond which it may begin to hallucinate. Investigating the perception of LLMs' KB is crucial for detecting hallucinations and LLMs' reliable generation. Current studies perceive LLMs' KB on questions with a concrete answer (close-ended questions) while paying limited attention to semi-open-ended questions (SoeQ) that correspond to many potential answers. Some researchers achieve it by judging whether the question is answerable or not. However, this paradigm is unsuitable for SoeQ, which are usually partially answerable, co…

6. Very close Review Cites this paper Not relevant

   Retrieval Augmentation Reduces Factual Errors in Knowledge-Intensive Language Model Tasks

   Dai Teng, Chang-Hao Zhang, Ji-Tong Zou · 2026 · Computer Life · Not summarized yet

   This review surveys retrieval-augmented generation for reducing factual errors, including adaptive and self-reflective approaches related to the paper’s retrieval analysis.

   Abstract

   Large language models (LLMs) have demonstrated exceptional capabilities across natural language processing (NLP) tasks; however, they remain persistently susceptible to generating factually incorrect content—a phenomenon broadly termed hallucination. Retrieval-augmented generation (RAG) has emerged as a principled paradigm for mitigating this limitation by grounding model outputs in dynamically retrieved external evidence, thereby substantially reducing factual errors in knowledge-intensive settings. This paper presents a comprehensive review of RAG research, tracing developments from early retrieval-enhanced pretraining frameworks to adaptive and self-reflective architectures. We examine ho…

7. Very close Review Cites this paper Not relevant

   Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

   Cunxiang Wang, Xiaoze Liu, Yuanhao Yue et al. · 2023 · Computation and Language · preprint, not peer reviewed · cited 306 times · AI summary available

   This survey covers LLM factuality, how models process facts, and evaluation methods, placing the paper’s knowledge-boundary study in a broader factuality overview.

   Abstract

   This survey addresses the crucial issue of factuality in Large Language Models (LLMs). As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital. We define the Factuality Issue as the probability of LLMs to produce content inconsistent with established facts. We first delve into the implications of these inaccuracies, highlighting the potential consequences and challenges posed by factual errors in LLM outputs. Subsequently, we analyze the mechanisms through which LLMs store and process facts, seeking the primary causes of factual errors. Our discussion then transitions to methodologies for evaluating LLM factuality, emphasizing key metrics,…

8. Very close Broader context Not relevant

   Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts

   Jian Xie, Kai Zhang, Jiangjie Chen et al. · 2023 · International Conference on Learning Representations · cited 479 times · Not summarized yet

   This study tests how LLMs respond when retrieved evidence conflicts with parametric knowledge, a neighbouring question about using external information.

   Abstract

   By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limitations of LLMs' static parametric memory. However, how receptive are LLMs to such external evidence, especially when the evidence conflicts with their parametric memory? We present the first comprehensive and controlled investigation into the behavior of LLMs when encountering knowledge conflicts. We propose a systematic framework to elicit high-quality parametric memory from LLMs and construct the corresponding counter-memory, which enables us to conduct a series of controlled experiments. Our investigation reveal…

Prior work it builds on

1. Very close Same question A method it uses Not relevant

   Do Large Language Models Know What They Don't Know?

   Zhangyue Yin, Qiushi Sun, Qipeng Guo et al. · 2023 · Annual Meeting of the Association for Computational Linguistics · cited 321 times · Not summarized yet

   This study evaluates whether LLMs recognize unanswerable questions, directly examining the self-knowledge that the paper investigates as factual-boundary awareness.

   Abstract

   Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks. Current research focuses on enhancing their performance within their existing knowledge. Despite their vast knowledge, LLMs are still limited by the amount of information they can accommodate and comprehend. Therefore, the ability to understand their own limitations on the unknows, referred to as self-knowledge, is of paramount importance. This study aims to evaluate LLMs' self-knowledge by assessing their ability to identify unanswerable or unknowable questions. We introduce an automated methodology to detect uncertainty in the responses of these models, prov…

2. Close Broader context A method it uses Not relevant

   Evaluating Open-Domain Question Answering in the Era of Large Language Models

   Ehsan Kamalloo, Nouha Dziri, C. Clarke et al. · 2023 · Annual Meeting of the Association for Computational Linguistics · cited 210 times · Not summarized yet

   This work evaluates open-domain QA answers from LLMs, addressing answer-quality measurement relevant to the paper’s QA performance analysis.

   Abstract

   Lexical matching remains the de facto evaluation method for open-domain question answering (QA). Unfortunately, lexical matching fails completely when a plausible candidate answer does not appear in the list of gold answers, which is increasingly the case as we shift from extractive to generative models. The recent success of large language models (LLMs) for QA aggravates lexical matching failures since candidate answers become longer, thereby making matching with the gold answers even more challenging. Without accurate evaluation, the true progress in open-domain QA remains unknown. In this paper, we conduct a thorough analysis of various open-domain QA models, including LLMs, by manually e…

3. Close Another approach to the problem Not relevant

   Generate rather than Retrieve: Large Language Models are Strong Context Generators

   Wenhao Yu, Dan Iter, Shuohang Wang et al. · 2022 · Computation and Language · cited 460 times · AI summary available

   GenRead replaces document retrieval with LLM-generated context for knowledge-intensive QA, offering a different way to supply external-style information.

   Abstract

   Knowledge-intensive tasks, such as open-domain question answering (QA), require access to a large amount of world or domain knowledge. A common approach for knowledge-intensive tasks is to employ a retrieve-then-read pipeline that first retrieves a handful of relevant contextual documents from an external corpus such as Wikipedia and then predicts an answer conditioned on the retrieved documents. In this paper, we present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given qu…

4. Close Groundwork for this paper Not relevant

   REALM: Retrieval-Augmented Language Model Pre-Training

   Kelvin Guu, Kenton Lee, Zora Tung et al. · 2020 · International Conference on Machine Learning · cited 3,602 times · Not summarized yet

   REALM introduces a retriever integrated with language-model pretraining and inference, providing an earlier retrieval-augmentation approach relevant to the paper.

   Abstract

   Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering. However, this knowledge is stored implicitly in the parameters of a neural network, requiring ever-larger networks to cover more facts. To capture knowledge in a more modular and interpretable way, we augment language model pre-training with a latent knowledge retriever, which allows the model to retrieve and attend over documents from a large corpus such as Wikipedia, used during pre-training, fine-tuning and inference. For the first time, we show how to pre-train such a knowledge retriever in an unsupervised manner, using masked language modeling a…

5. Close Broader context A method it uses Not relevant

   Automatic Evaluation of Attribution by Large Language Models

   Xiang Yue, Boshi Wang, Kai Zhang et al. · 2023 · Conference on Empirical Methods in Natural Language Processing · cited 97 times · Not summarized yet

   This work evaluates whether LLM-generated claims are supported by cited references, examining factual reliability in retrieval-grounded generation.

   Abstract

   A recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external references to generate and support its claims. However, evaluating the attribution, i.e., verifying whether the generated statement is fully supported by the cited reference, remains an open problem. Although human evaluation is common practice, it is costly and time-consuming. In this paper, we investigate the automatic evaluation of attribution given by LLMs. We begin by defining different types of attribution errors, and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. The fine-tuning data is repurposed from rela…

Papers that cite it

1. Very close Same question Uses this paper in depth Not relevant

   When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation

   Shi-Yu Ni, Ke-Ping Bi, J. Guo et al. · 2024 · Annual Meeting of the Association for Computational Linguistics · cited 87 times · Not summarized yet

   This work measures LLM overconfidence and asks when retrieval is needed, directly studying knowledge-boundary awareness as a guide to retrieval.

   Abstract

   Large Language Models (LLMs) have been found to have difficulty knowing they do not possess certain knowledge and tend to provide specious answers in such cases. Retrieval Augmentation (RA) has been extensively studied to mitigate LLMs' hallucinations. However, due to the extra overhead and unassured quality of retrieval, it may not be optimal to conduct RA all the time. A straightforward idea is to only conduct retrieval when LLMs are uncertain about a question. This motivates us to enhance the LLMs' ability to perceive their knowledge boundaries to help RA. In this paper, we first quantitatively measure LLMs' such ability and confirm their overconfidence. Then, we study how LLMs' certainty…

2. Very close Broader context Uses this paper in depth Not relevant

   Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation

   Yu-Hao Wang, Ruiyang Ren, Yucheng Wang et al. · 2025 · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · cited 10 times · Not summarized yet

   This study investigates how LLMs combine parametric and retrieved knowledge in RAG, a neighbouring aspect of the paper’s retrieval-augmentation analysis.

   Abstract

   Considering the inherent limitations of parametric knowledge in large language models (LLMs), retrieval-augmented generation (RAG) is widely employed to expand their knowledge scope. Since RAG has shown promise in knowledge-intensive tasks like open-domain question answering, its broader application to complex tasks and intelligent assistants has further advanced its utility. Despite this progress, the underlying knowledge utilization mechanisms of LLM-based RAG remain underexplored. In this paper, we present a systematic investigation of the intrinsic mechanisms by which LLMs integrate internal (parametric) and external (retrieved) knowledge in RAG scenarios. Specially, we employ knowledge …

3. Close Broader context Uses this paper in depth Not relevant

   The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models

   J. Li, Jie Chen, Ruiyang Ren et al. · 2024 · Annual Meeting of the Association for Computational Linguistics · cited 181 times · Not summarized yet

   This empirical study examines hallucination detection, sources, and mitigation, broader factual-reliability questions related to the paper’s confidence and retrieval findings.

   Abstract

   In the era of large language models (LLMs), hallucination (i.e., the tendency to generate factually incorrect content) poses great challenge to trustworthy and reliable deployment of LLMs in real-world applications. To tackle the LLM hallucination, three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them (mitigation). To address these challenges, this work presents a systematic empirical study on LLM hallucination, focused on the the three aspects of hallucination detection, source and mitigation. Specially, we construct a new hallucination benchmark HaluEval 2.0, and designs a simple yet eff…

4. Very close Review Not relevant

   Retrieval-Augmented Generation for Large Language Models: A Survey

   Yunfan Gao, Yun Xiong, Xinyu Gao et al. · 2023 · Computation and Language · preprint, not peer reviewed · cited 4,421 times · AI summary available

   This survey reviews RAG methods and their effects on answer accuracy, hallucination, and knowledge updates, covering the retrieval setting examined in the paper.

   Abstract

   Large language models (LLMs) demonstrate powerful capabilities, but they still face challenges in practical applications, such as hallucinations, slow knowledge updates, and lack of transparency in answers. Retrieval-Augmented Generation (RAG) refers to the retrieval of relevant information from external knowledge bases before answering questions with LLMs. RAG has been demonstrated to significantly enhance answer accuracy, reduce model hallucination, particularly for knowledge-intensive tasks. By citing sources, users can verify the accuracy of answers and increase trust in model outputs. It also facilitates knowledge updates and the introduction of domain-specific knowledge. RAG effectivel…

5. Close Same method, another problem Uses this paper in depth Not relevant

   UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models

   Bo-Yang Xue, Fei Mi, Qi Zhu et al. · 2024 · Annual Meeting of the Association for Computational Linguistics · cited 13 times · Not summarized yet

   UAlign uses uncertainty estimates to represent LLM knowledge boundaries and improve factual answers, applying boundary perception to factuality alignment.

   Abstract

   Despite demonstrating impressive capabilities, Large Language Models (LLMs) still often struggle to accurately express the factual knowledge they possess, especially in cases where the LLMs' knowledge boundaries are ambiguous. To improve LLMs' factual expressions, we propose the UAlign framework, which leverages Uncertainty estimations to represent knowledge boundaries, and then explicitly incorporates these representations as input features into prompts for LLMs to Align with factual knowledge. First, we prepare the dataset on knowledge question-answering (QA) samples by calculating two uncertainty estimations, including confidence score and semantic entropy, to represent the knowledge boun…