# A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia

**Authors:** Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kıcıman, Hamid Palangi, Barun Patra, Robert West  
**arXiv:** [2312.02073](https://arxiv.org/abs/2312.02073)  
**Date:** 2023-12-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.02073v1/

## Key points

- Large language models (LLMs) have remarkable abilities in storing and retrieving factual knowledge and adapting to new contextual information.
- The mechanisms behind LLMs' ability to ground themselves in context when faced with contradictory information are not fully understood.
- Fakepedia is a counterfactual dataset introduced in this study to evaluate LLMs' grounding abilities.
- GPT-4-turbo tends to rely heavily on its parametric knowledge, while Mistral-7B consistently chooses the grounded answer according to the benchmarking results.
- Causal mediation analysis on LLM components shows that LLM grounding can be predicted with an accuracy of 92.8% by inspecting the computational graph alone.
- Only a few Multilayer Perceptrons (MLPs) in the Transformer can predict non-grounded behavior effectively.
- This study provides a coherent understanding of how grounding and factual recall mechanisms interact within LLMs.

## Summary

Large language models (LLMs) have shown remarkable abilities in storing and retrieving factual knowledge, as well as adapting to new contextual information. However, the mechanisms behind their ability to ground themselves in context are still not fully understood, particularly when the information in context contradicts the factual knowledge embedded in the model's parameters. This issue is crucial for retrieval-augmented generation methods that aim to enhance context with up-to-date information, hoping that grounding can rectify outdated parametric knowledge. To address this gap, the authors of this study introduce Fakepedia, a counterfactual dataset designed to evaluate the grounding abilities of LLMs when there is a clash between parametric knowledge and in-context information. They benchmark various LLMs using Fakepedia and find that GPT-4-turbo tends to rely heavily on its parametric knowledge, while Mistral-7B consistently chooses the grounded answer. The researchers then conduct causal mediation analysis on LLM components when answering Fakepedia queries. They demonstrate that by inspecting the computational graph alone, LLM grounding can be predicted with an impressive accuracy of 92.8%. This prediction is particularly effective because only a few Multilayer Perceptrons (MLPs) in the Transformer can predict non-grounded behavior. These findings, combined with existing research on factual recall mechanisms, provide a coherent understanding of how grounding and factual recall mechanisms interact within LLMs. Overall, this study sheds light on the underlying processes that enable LLMs to adapt and ground themselves in context while recalling relevant information.