# The Vector Grounding Problem

**Authors:** Dimitri Coelho Mollo, Raphaël Millière  
**arXiv:** [2304.01481](https://arxiv.org/abs/2304.01481)  
**Date:** 2023-04-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.01481v1/

## Key points

- Artificial intelligence has made remarkable advancements in the past decade due to deep learning techniques.
- Large Language Models (LLMs) are neural networks trained on linguistic data using the Transformer architecture and self-attention mechanism.
- LLMs such as BERT, GPT-3, and PaLM have demonstrated near-human or super-human performance across numerous linguistic tasks.
- The Symbol Grounding Problem arises when AI systems generate outputs without direct interaction with the world.
- Five distinct notions of grounding in biological or artificial systems are referential, sensorimotor, relational, communicative, and epistemic grounding.
- Referential grounding is at the heart of what is called the Vector Grounding Problem for modern LLMs.
- Fine-tuning LLMs with Reinforcement Learning from Human Feedback can overcome this problem by establishing causal-historical relations to the world that underpin intrinsic meaning.
- Multimodality and embodiment are neither necessary nor sufficient conditions for referential grounding in artificial systems.
- LLMs learn distributed representations of words conditioned on their contexts in training data and encode each token as a vector in a high-dimensional space.

## Summary

Over the past decade, the field of artificial intelligence has made remarkable advancements, largely due to the emergence and maturation of deep learning techniques. One notable development within the domain of natural language processing has been the creation of massive neural networks trained on an abundance of linguistic data, known as Large Language Models (LLMs). These LLMs are built upon the Transformer architecture, which features a mechanism called self-attention that models long-range dependencies between linguistic elements across extensive text sequences. LLMs such as BERT, GPT-3, and PaLM have showcased remarkable capabilities in generating coherent and contextually relevant texts spanning a diverse range of subjects from current events to philosophical essays; answering commonsense questions; explaining novel jokes; and consistently demonstrating near-human or even super-human performance across numerous linguistic tasks. The exceptional performance of LLMs raises intriguing questions regarding the nature of language processing, language understanding, and linguistic acts such as writing or speaking. One issue that arises is how AI systems can generate meaningful outputs when they have no direct interaction with the world. This problem is known as the Symbol Grounding Problem. In this paper, we revisit this problem and explore its implications for contemporary approaches to artificial language modelling. We differentiate various ways in which internal representations can be grounded in biological or artificial systems, identifying five distinct notions discussed in literature: referential, sensorimotor, relational, communicative, and epistemic grounding. We clarify the differences between these notions and argue that referential grounding lies at the heart of what we call the Vector Grounding Problem for modern LLMs. Drawing on theories of representational content in philosophy and cognitive science, we propose that certain LLMs fine-tuned with Reinforcement Learning from Human Feedback possess necessary features to overcome this problem because they stand in causal-historical relations to the world that underpin intrinsic meaning. Furthermore, we argue that multimodality and embodiment are neither necessary nor sufficient conditions for referential grounding in artificial systems. LLMs learn distributed representations of words conditioned on the contexts in which they appear in the training data. These models process input text into a sequence of tokens and encode each token as a vector of real-valued numbers in a high-dimensional vector space. The dimensions of these vectors are tuned by the model to best predict surrounding tokens and capture distributional patterns with semantically similar tokens having vectors close together in this high dimensional space. Overall, this paper aims to make modest progress toward understanding capabilities of LLMs and interpreting their outputs by revisiting Symbol Grounding Problem proposing solutions to overcome it so as to shed light on how LLMs can generate meaningful outputs despite not having direct interaction with world.