# An LLM-based Recommender System Environment

**Authors:** Nathan Corecco, Giorgio Piatti, Luca A. Lanzendörfer, Flint Xiaofeng Fan, Roger Wattenhofer  
**arXiv:** [2406.01631](https://arxiv.org/abs/2406.01631)  
**Date:** 2024-06-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.01631v1/

## Key points

- Reinforcement learning (RL) is a popular approach in recommender systems for optimizing long-term rewards and enhancing user experiences.
- Challenges in implementing RL include limited availability of online data for training on-policy methods, requiring costly human interaction for model training.
- A comprehensive framework has been proposed that leverages synthetic environments and large language models (LLMs) to effectively train RL-based recommender systems by simulating human behavior.
- The framework introduces a modular and innovative approach to model training using LLMs as synthetic users like Emily Johnson, a 37-year-old detective with specific preferences.
- MovieLens and Amazon Book Dataset subsets are used for recommendations, with items retrieved based on similarity to query items using Sentence-T5 embeddings and cosine distance calculations.
- The LLM generates ratings based on prompts constructed from user descriptions, optimizing performance through few-shot prompting techniques.
- Ablation experiments validate the standard configuration choices made in the framework setup.
- Results show improvements in average reward, personalization within recommendations, and reduction in disliked genres percentage compared to traditional approaches like DQN, PPO, TRPO, and A2C.

## Summary

Recommender Systems: Enhancing Efficiency and Personalization through Synthetic Environments and Large Language Models

In the realm of recommender systems, reinforcement learning (RL) has emerged as a popular approach for optimizing long-term rewards and enhancing user experiences. However, implementing RL in recommender systems poses challenges such as limited availability of online data for training on-policy methods, necessitating costly human interaction for model training. To address these issues, a comprehensive framework for synthetic environments leveraging large language models (LLMs) has been proposed. This framework simulates human behavior to train RL-based recommender systems effectively. By utilizing LLMs as synthetic users, the framework introduces a modular and innovative approach to model training. The individual generated within this context is Emily Johnson, a 37-year-old detective with a passion for collecting compact discs. In her leisure time, Emily enjoys watching romance and horror movies but tends to avoid action and comedy genres due to finding them chaotic and uninteresting. Her secondary hobbies include reading mystery novels and playing the piano. The framework utilizes MovieLens for movie data and a subset of the Amazon Book Dataset for book recommendations. The setup involves retrieving items based on similarity to query items using Sentence-T5 embeddings and cosine distance calculations. The LLM generates ratings based on prompts constructed from user descriptions, optimizing performance through few-shot prompting techniques. Ablation experiments justify the standard configuration choices made in the framework setup. Results from RL methods trained on this framework show improvements in average reward, personalization within recommendations, and reduction in disliked genres percentage compared to traditional approaches like DQN, PPO, TRPO, and A2C. Overall,this refined summary highlights the innovative use of synthetic environments powered by LLMs to enhance RL-based recommender systems' efficiency and effectiveness in providing personalized recommendations tailored to individual preferences like those of Emily Johnson.