# Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

**Authors:** Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, Jeff Clune  
**arXiv:** [2206.11795](https://arxiv.org/abs/2206.11795)  
**Date:** 2022-06-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.11795v1/

## Key points

- Pretraining on noisy, internet-scale datasets is extensively studied for training models with broad, general capabilities for text, images, and other modalities.
- Publicly available data in sequential decision domains often lacks the labels required to train behavioral priors in the same way.
- A team of researchers led by Bowen Baker developed a novel approach called Video PreTraining (VPT), which extends the internet-scale pretraining paradigm to sequential decision domains through semi-supervised imitation learning.
- In VPT, agents learn to act by watching online unlabeled videos.
- With a small amount of labeled data, they could train an inverse dynamics model accurate enough to label a huge unlabeled source of online data from which they could then train a general behavioral prior.
- The team's models exhibited human-level performance for many tasks and were even able to craft diamond tools in Minecraft - a task that can take proficient humans upwards of 20 minutes (24,000 environment actions) of gameplay to accomplish.
- VPT represents an important step forward in developing more efficient and effective methods for training agents in sequential decision domains where labeled data is scarce or nonexistent.

## Summary

The technique of pretraining on noisy, internet-scale datasets has been extensively studied for training models with broad, general capabilities for text, images, and other modalities. However, in sequential decision domains such as robotics, video games, and computer use, publicly available data often lacks the labels required to train behavioral priors in the same way. To address this issue, a team of researchers led by Bowen Baker developed a novel approach called Video PreTraining (VPT), which extends the internet-scale pretraining paradigm to sequential decision domains through semi-supervised imitation learning. In VPT, agents learn to act by watching online unlabeled videos. The researchers showed that with a small amount of labeled data they could train an inverse dynamics model accurate enough to label a huge unlabeled source of online data - specifically online videos of people playing Minecraft - from which they could then train a general behavioral prior. Despite using the native human interface (mouse and keyboard at 20Hz), the behavioral prior exhibited nontrivial zero-shot capabilities and could be fine-tuned with both imitation learning and reinforcement learning to hard-exploration tasks that are impossible to learn from scratch via reinforcement learning. The team's models exhibited human-level performance for many tasks and were even able to craft diamond tools in Minecraft - a task that can take proficient humans upwards of 20 minutes (24,000 environment actions) of gameplay to accomplish. This achievement is significant because it demonstrates that VPT can enable agents to learn complex behaviors from raw sensory inputs without explicit supervision or reward signals. Overall, VPT represents an important step forward in developing more efficient and effective methods for training agents in sequential decision domains where labeled data is scarce or nonexistent. The researchers' findings have implications not only for video game AI but also for robotics and other applications where autonomous agents must make decisions based on sensory inputs.