# Monitoring Latent World States in Language Models with Propositional Probes

**Authors:** Jiahai Feng, Stuart Russell, Jacob Steinhardt  
**arXiv:** [2406.19501](https://arxiv.org/abs/2406.19501)  
**Date:** 2024-06-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.19501v1/

## Key points

- Researchers focus on monitoring latent world states in language models using propositional probes
- Language models exhibit biases and unfaithful responses, prompting the need for interpretability tools
- Hypothesis: Language models encode input contexts in a latent world model
- Propositional probes compose tokens for lexical information and bind them into logical propositions representing the world state
- Introduction of a Hessian-based algorithm to identify the binding subspace, inspired by previous work using Jacobians
- Evaluation through quantitative causal interventions and qualitative analysis
- Effectiveness of propositional probes demonstrated in a closed-world setting with finite predicates and properties
- Probes successfully generalize to more complex scenarios such as short stories and translations into different languages like Spanish
- Exploration of scenarios where language models respond unfaithfully to input contexts, including prompt injections, backdoor attacks, and gender bias
- Decoded propositions remain faithful even when language models respond unfaithfully, suggesting accurate encoding but struggles with faithful decoding

## Summary

In this study, the researchers focus on monitoring latent world states in language models using propositional probes. Language models are known to exhibit biases and unfaithful responses, prompting the need for interpretability tools to correct such behavior. The researchers hypothesize that language models encode input contexts in a latent world model and aim to extract this information through propositional probes. These probes compose tokens for lexical information and bind them into logical propositions representing the world state. To identify the binding subspace, a Hessian-based algorithm is introduced, inspired by previous work using Jacobians in unrelated tasks. This algorithm is evaluated through quantitative causal interventions and qualitative analysis. The researchers demonstrate the effectiveness of propositional probes in a closed-world setting with finite predicates and properties. Despite being trained on simple templated contexts, these probes successfully generalize to more complex scenarios such as short stories and translations into different languages like Spanish. Furthermore, the study explores various scenarios where language models respond unfaithfully to input contexts, including prompt injections, backdoor attacks, and gender bias. Interestingly, even in these cases, the decoded propositions remain faithful, suggesting that language models often encode an accurate world model but may struggle with faithful decoding.