# Generator and Critic: A Deep Reinforcement Learning Approach for Slate Re-ranking in E-commerce

**Authors:** Jianxiong Wei, Anxiang Zeng, Yueqiu Wu, Peng Guo, Qingsong Hua, Qingpeng Cai  
**arXiv:** [2005.12206](https://arxiv.org/abs/2005.12206)  
**Date:** 2020-05-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2005.12206v1/

## Key points

- The slate re-ranking problem in e-commerce aims to improve user satisfaction by considering relationships between items
- Challenges include complex slate evaluation and the vast action space for finding the best slate
- A novel Generator and Critic approach is introduced for slate re-ranking
- The Critic component evaluates slates, while the Generator ranks items using reinforcement learning techniques
- The Full Slate Critic (FSC) model is proposed to account for real user preferences and mitigate biases
- A new exploration reinforcement learning algorithm called PPO-Exploration addresses the issue of a vast action space
- Experimental results show that the FSC model outperforms current methods significantly, and PPO-Exploration surpasses existing algorithms by a substantial margin
- Live experiments on a large e-commerce platform demonstrate improved efficiency (4% increase in GMV and 5% rise in number of orders) and diversity with the Generator and Critic approach

## Summary

, , , , 

In the realm of e-commerce, the slate re-ranking problem seeks to enhance user satisfaction by considering the intricate interplay between items, as opposed to traditional point-wise ranking methods. Previous research has explored various approaches, such as end-to-end models or scoring functions that balance individual item scores with overall diversity. However, two key challenges remain largely unaddressed: evaluating a slate is complex due to the nuanced relationships between items within it; and even with optimal evaluation criteria, finding the best slate is daunting given the exponentially large action space. To tackle these challenges, this paper introduces a novel Generator and Critic approach for slate re-ranking in e-commerce. The <b>Critic component evaluates slates,</b> while the <b>Generator ranks items using reinforcement learning techniques.</b> A Full Slate Critic (FSC) model is proposed to account for real user preferences and mitigate biases present in existing models. To address the issue of a vast action space, a new exploration reinforcement learning algorithm called PPO-Exploration is introduced. Experimental results demonstrate that the FSC model outperforms current state-of-the-art slate evaluation methods significantly, while the PPO-Exploration algorithm surpasses existing reinforcement learning approaches by a substantial margin. Through live experiments on one of the largest e-commerce platforms globally, the Generator and Critic approach proves its efficacy by improving both slate efficiency (4% increase in GMV and 5% rise in number of orders) and diversity. This innovative methodology showcases promising potential for enhancing user experiences and optimizing recommendations in e-commerce settings.