# A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress

**Authors:** Saurabh Arora, Prashant Doshi  
**arXiv:** [1806.06877](https://arxiv.org/abs/1806.06877)  
**Date:** 2018-06-18  
**Summary page:** https://www.summarizepaper.com/en/paper/1806.06877v1/

## Key points

- Inverse reinforcement learning (IRL) involves inferring the reward function of an observed agent based on its policy or behavior
- This article serves as a comprehensive survey of the current literature in IRL
- The survey highlights challenges in IRL, including accurate inference of the reward function, generalizability of learned models, correctness of prior knowledge, and increased solution complexity with problem size
- Current methods are discussed for addressing these challenges and mitigating their impact
- Extensions to traditional IRL methods are explored, such as handling inaccurate and incomplete perception, dealing with incomplete models, accommodating multiple rewards, and incorporating non-linear reward functions
- Insights are provided on how these extensions can be addressed within the framework of IRL
- The survey concludes with an overview of broad advances in the research area of IRL and highlights open research questions for further exploration

## Summary

Inverse reinforcement learning (IRL) is a problem in machine learning that involves inferring the reward function of an observed agent based on its policy or behavior. This article, written by Saurabh Arora and Prashant Doshi, serves as a comprehensive survey of the current literature in IRL. It aims to provide researchers and practitioners with a reference to understand the challenges associated with IRL and select appropriate approaches for their specific problems. The survey begins by formally introducing the IRL problem and highlighting its central challenges. These challenges include accurate inference of the reward function, generalizability of learned models, correctness of prior knowledge, and the increase in solution complexity as problem size grows. The authors discuss how current methods address these challenges and mitigate their impact. Additionally, the article explores extensions to traditional IRL methods such as handling inaccurate and incomplete perception, dealing with incomplete models, accommodating multiple rewards, and incorporating non-linear reward functions. The authors provide insights into how these extensions can be addressed within the framework of IRL. The discussion concludes with an overview of broad advances in the research area of IRL and highlights currently open research questions that warrant further exploration. By presenting a comprehensive overview of existing literature, this survey serves as a valuable resource for researchers and practitioners interested in understanding the nuances of IRL and selecting appropriate methodologies for their specific applications.