# Generating Query-Relevant Document Summaries via Reinforcement Learning

**Authors:** Nitin Yadav, Changsung Kang, Hongwei Shang, Ming Sun  
**arXiv:** [2508.08404](https://arxiv.org/abs/2508.08404)  
**Date:** 2025-08-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2508.08404v1/

## Key points

- Traditional reliance on product titles for ranking models in e-commerce search engines is insufficient in capturing query intent and delivering optimal relevance predictions.
- Product descriptions are often too verbose and lengthy for real-time ranking, leading to challenges in achieving search relevance.
- The novel reinforcement learning framework ReLSum aims to generate concise and query-relevant summaries of product descriptions efficiently.
- ReLSum leverages relevance scores as rewards to align summarization and ranking objectives effectively, addressing misaligned learning targets seen in previous methods.
- The framework utilizes a trainable large language model (LLM) to produce summaries that enhance the performance of cross-encoder ranking models.
- ReLSum has shown significant improvements in offline metrics such as recall and NDCG, as well as online user engagement metrics.
- Generating single summaries for each document or product through reinforcement learning can enhance performance on downstream tasks like search relevance.
- Strategies for constructing training datasets have been included to facilitate efficient fine-tuning of summary-generating models within the ReLSum framework.
- The paper compares Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) methodologies for text summarization purposes, contributing to optimizing text summarization processes using advanced approaches like LLMs in zero-shot and few-shot settings.

## Summary

In the realm of e-commerce search engines, traditional reliance on product titles for ranking models has proven insufficient in capturing query intent and delivering optimal relevance predictions. This is due to the fact that product descriptions, while richer in information, are often too verbose and lengthy for real-time ranking. This issue is further exacerbated by computationally expensive architectures like cross-encoder models. To address this challenge, a novel reinforcement learning framework called ReLSum has been introduced. This framework aims to generate concise and query-relevant summaries of product descriptions to enhance search relevance efficiently. By leveraging relevance scores as rewards to align summarization and ranking objectives effectively, ReLSum addresses issues such as misaligned learning targets seen in previous methods. One key aspect of the ReLSum framework is its use of a trainable large language model (LLM) to produce summaries that serve as input for cross-encoder ranking models. This approach has shown significant improvements in offline metrics such as recall and NDCG, as well as online user engagement metrics. The paper also delves into the concept of generating single summaries for each document or product through reinforcement learning with the ultimate goal of enhancing performance on downstream tasks such as search relevance. To facilitate efficient fine-tuning of summary-generating models within this framework, strategies for constructing training datasets have been included. Additionally, the paper compares two methodologies - Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) - for summarization purposes. The exploration of these techniques contributes to advancing the field by optimizing text summarization processes using cutting-edge approaches like LLMs in zero-shot and few-shot settings. Overall, this comprehensive study not only sheds light on the challenges faced by e-commerce search engines but also presents innovative solutions through advanced reinforcement learning frameworks tailored for summarization tasks. By improving the quality of generated summaries and aligning them with ranking objectives, this research paves the way for enhanced search relevance in large-scale e-commerce systems while balancing performance with latency constraints effectively.