# Generative Adversarial Imitation Learning

**Authors:** Jonathan Ho, Stefano Ermon  
**arXiv:** [1606.03476](https://arxiv.org/abs/1606.03476)  
**Date:** 2016-06-10  
**Summary page:** https://www.summarizepaper.com/en/paper/1606.03476v1/

## Key points

- The paper explores learning a policy from expert behavior without direct interaction or reinforcement signals
- Traditional approach involves recovering expert's cost function and then extracting a policy through reinforcement learning
- Authors propose a new framework that directly extracts a policy from data using inverse reinforcement learning
- Framework draws an analogy between imitation learning and generative adversarial networks (GANs)
- Model-free imitation learning algorithm developed within the framework shows significant performance gains over existing methods
- Approach eliminates the need for costly interactions with experts or access to reinforcement signals
- More efficient and practical for applications with limited resources
- Paper presents a novel framework for policy extraction and introduces an effective algorithm for imitation learning
- Results suggest promising potential for advancing research in complex behavior imitation in various domains.

## Summary

The paper titled "Generative Adversarial Imitation Learning" by Jonathan Ho and Stefano Ermon explores the problem of learning a policy from expert behavior without direct interaction with the expert or access to reinforcement signals. The traditional approach involves recovering the expert's cost function using inverse reinforcement learning and then extracting a policy through reinforcement learning. However, this method is indirect and can be slow. To address this issue, the authors propose a new framework that directly extracts a policy from data as if it were obtained through reinforcement learning following inverse reinforcement learning. They demonstrate that their framework draws an analogy between imitation learning and generative adversarial networks (GANs). Leveraging this analogy, they develop a model-free imitation learning algorithm within their framework. The algorithm they propose shows significant performance gains over existing model-free methods when imitating complex behaviors in large, high-dimensional environments. By directly extracting policies from data, their approach eliminates the need for costly interactions with experts or access to reinforcement signals. This makes it more efficient and practical for applications where such resources may be limited. Overall, the paper presents a novel framework for policy extraction and introduces an effective algorithm for imitation learning. The results suggest that their approach has promising potential for advancing research in complex behavior imitation in various domains.