# Grounded Language Learning in a Simulated 3D World

**Authors:** Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, Marcus Wainwright, Chris Apps, Demis Hassabis, Phil Blunsom  
**arXiv:** [1706.06551](https://arxiv.org/abs/1706.06551)  
**Date:** 2017-06-20  
**Summary page:** https://www.summarizepaper.com/en/paper/1706.06551v2/

## Key points

- Artificially intelligent technology surrounds us, making decisions and taking actions on our behalf
- Effective communication between humans and artificial agents is crucial
- Human language is the most natural and compelling means of interaction
- Researchers have developed an agent that learns to interpret language in a simulated 3D environment
- The agent is rewarded for successfully executing written instructions in the virtual world
- Combination of reinforcement learning and unsupervised learning techniques used
- Agent starts with minimal prior knowledge and gradually learns to associate language with physical surroundings and actions
- Agent can comprehend familiar language in unfamiliar situations and interpret novel instructions
- Agent's semantic knowledge grows, enabling faster learning of new words and showcasing capacity for generalizing semantic knowledge
- Research demonstrates potential of using grounded approach to reconcile ambiguous natural language with complexity of physical world
- Enabling AI agents to understand human language in an embodied manner paves way for more effective communication between humans and machines.

## Summary

In today's world, we are surrounded by artificially intelligent technology that makes decisions and takes actions on our behalf. This creates a need for effective communication between humans and artificial agents, with human language being the most natural and compelling means of interaction. To address this challenge, researchers have developed an agent that learns to interpret language in a simulated 3D environment. The agent is rewarded for successfully executing written instructions within this virtual world. Through a combination of reinforcement learning and unsupervised learning techniques, the agent starts with minimal prior knowledge and gradually learns to associate linguistic symbols with perceptual representations of its physical surroundings and sequences of actions. What sets this agent apart is its ability to comprehend language beyond its prior experience. It can apply familiar language to unfamiliar situations and even interpret entirely novel instructions. As the agent's semantic knowledge grows, it becomes faster at learning new words, showcasing its capacity for generalizing and bootstrapping semantic knowledge. This research demonstrates the potential of using a grounded approach to reconcile ambiguous natural language with the complexity of the physical world. By enabling AI agents to understand human language in a more embodied manner, we can pave the way for more effective communication between humans and machines in various domains.