# Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks

**Authors:** B. A. Levinstein, Daniel A. Herrmann  
**arXiv:** [2307.00175](https://arxiv.org/abs/2307.00175)  
**Date:** 2023-06-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.00175v1/

## Key points

- Large language models (LLMs) and their beliefs
- Evaluation of existing approaches for measuring LLM beliefs
- Conceptual limitations of current measurement methods
- Questioning whether LLMs should be expected to have beliefs
- Refuting arguments against LLMs having beliefs
- Emphasizing the empirical nature of determining LLM beliefs
- Overview of transformer architecture, specifically autoregressive, decoder-only models like GPT and LLaMA series
- Aim of the article: shedding light on LLM belief status and suggesting future research paths

## Summary

This article explores the question of whether large language models (LLMs) have beliefs and how we can measure them. The authors evaluate two existing approaches for measuring LLM beliefs but find that these methods fail to generalize effectively. They argue that even if LLMs have beliefs, these measurement methods are unlikely to be successful due to conceptual reasons. The authors then consider whether LLMs should be expected to have beliefs in the first place and address arguments suggesting that they cannot. They refute these arguments by pointing out a philosophical mistake and providing a more productive framing of the belief status in LLMs. Moreover, they emphasize that determining whether LLMs have beliefs is an empirical matter. The article also provides an overview of transformer architecture, specifically focusing on autoregressive, decoder-only models like OpenAI's GPT series and Meta's LLaMA series. Overall, this article aims to shed light on the status of beliefs in LLMs and suggests potential paths for future research in this area.