# Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation

**Authors:** Xueying Bai, Jian Guan, Hongning Wang  
**arXiv:** [1911.03845](https://arxiv.org/abs/1911.03845)  
**Date:** 2019-11-10  
**Summary page:** https://www.summarizepaper.com/en/paper/1911.03845v3/

## Key points

- Authors Xueying Bai, Jian Guan, and Hongning Wang explore reinforcement learning for optimizing policies in recommender systems
- Current solutions mainly focus on model-free approaches, leading to high costs for model learning
- The authors propose a novel model-based reinforcement learning solution using a generative adversarial network
- Incorporation of a discriminator helps evaluate the quality of generated data and scale resulting rewards
- This approach mitigates bias in learned models and policies, improving policy learning from offline and generated data sources
- Empirical evaluations support the effectiveness of their solution in optimizing policies within recommender systems
- The research bridges the gap between offline evaluation methods and challenges posed by large action spaces
- By combining model-based reinforcement learning with adversarial training, the authors offer a promising avenue for enhancing policy optimization while reducing reliance on costly real-time interactions

## Summary

In their paper titled "Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation," authors Xueying Bai, Jian Guan, and Hongning Wang explore reinforcement learning as a powerful tool for optimizing policies in recommender systems. They address the prevalent focus on model-free approaches in current solutions, which require frequent interactions with the real environment and result in high costs for model learning. To overcome these limitations, the authors propose a novel model-based reinforcement learning solution that utilizes a generative adversarial network to model user-agent interaction. One key aspect of their approach is the incorporation of a discriminator to evaluate the quality of generated data and scale resulting rewards. This helps mitigate bias in both the learned model and policy, ultimately improving policy learning from offline and generated data sources. The authors support their theoretical framework with empirical evaluations that demonstrate the effectiveness of their solution in optimizing policies within recommender systems. By bridging the gap between offline evaluation methods like importance sampling and challenges posed by large action spaces, this research offers valuable insights into advancing reinforcement learning techniques for online recommendation systems. Through their innovative approach combining model-based reinforcement learning with adversarial training, Bai, Guan, and Wang provide a promising avenue for enhancing policy optimization in recommender systems while reducing reliance on costly real-time interactions with the environment.