# SPAR: Personalized Content-Based Recommendation via Long Engagement Attention

**Authors:** Chiyu Zhang, Yifei Sun, Jun Chen, Jie Lei, Muhammad Abdul-Mageed, Sinong Wang, Rong Jin, Sem Park, Ning Yao, Bo Long  
**arXiv:** [2402.10555](https://arxiv.org/abs/2402.10555)  
**Date:** 2024-02-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.10555v1/

## Key points

- Study focuses on content-based recommendation systems using text features for personalized recommendations
- Framework designed for text content can be integrated into broader recommendation systems with non-content related features
- Experiments used base-sized encoder model for computational efficiency and model performance
- Ethical considerations addressed regarding the use of large language models (LLMs) in generating user interest summaries
- Ongoing research aims to improve social fairness, accuracy, and reliability of LLMs in recommendation systems
- Publicly available datasets with anonymized user identities used to protect privacy
- Importance of content-based recommendation systems in addressing long-tailed and cold-start problems highlighted
- Pretrained language models (PLMs) have improved NLP tasks and incorporated into recommendation systems
- Challenges remain in effectively encoding long user engagement histories
- Introduction of SPAR framework leveraging PLMs, poly-attention layers, and attention sparsity mechanisms for session-based encoding
- SPAR demonstrates superior performance compared to existing state-of-the-art methods on benchmark datasets

## Summary

This study focuses on content-based recommendation systems that use text features as the primary input for personalized content recommendations. The developed framework is specifically designed for text content but can also be integrated into a broader recommendation system that incorporates non-content related sparse/dense features or content from other modalities. The experiments conducted used a base-sized encoder model to balance computational efficiency and model performance; however, future work will explore the use of smaller-sized models to optimize performance and efficiency in real-world applications. Ethical considerations are addressed regarding the use of large language models (LLMs) to generate summaries of user interests. It is acknowledged that LLM outputs may reflect societal biases or inaccuracies known as hallucinations, which could impact the recommendations generated by models utilizing these summaries. However, ongoing research aimed at improving the social fairness, accuracy, and reliability of LLMs is expected to enhance the performance and ethical standards of recommendation systems incorporating LLMs. Additionally, publicly available datasets with anonymized user identities were used in the experiments to protect privacy. The introduction highlights the importance of content-based recommendation systems in addressing long-tailed and cold-start problems commonly encountered in traditional ID-based recommendation systems by leveraging textual content for more precise and personalized recommendations. Pretrained language models (PLMs) have significantly improved NLP tasks and have been successfully incorporated into recommendation systems for encoding textual inputs. Despite these advancements, challenges remain in effectively encoding long user engagement histories due to their extensive nature. In response to these challenges, this paper introduces a novel content-based recommendation framework called SPAR that leverages PLMs, poly-attention layers, and attention sparsity mechanisms to encode user histories in a session-based manner. By combining user and item side features for engagement prediction while maintaining standalone representations for both sides, SPAR demonstrates superior performance compared to existing state-of-the-art methods on benchmark datasets.