# Monolith: Real Time Recommendation System With Collisionless Embedding Table

**Authors:** Zhuoran Liu, Leqi Zou, Xuan Zou, Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, Youlong Cheng  
**arXiv:** [2209.07663](https://arxiv.org/abs/2209.07663)  
**Date:** 2022-09-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2209.07663v1/

## Key points

- Importance of building a scalable and real-time recommendation system for time-sensitive customer feedback
- Deep learning frameworks like TensorFlow or PyTorch are not suitable for recommendation scenarios due to static parameters and dense computations
- Introduction of Monolith, a system specifically designed for online training in recommendation systems
- Monolith's design is driven by observations of application workloads and production environments, setting it apart from other recommendation systems
- Contributions of Monolith include collisionless embedding table with optimizations like expirable embeddings and frequency filtering to reduce memory footprint
- Production-ready online training architecture with high fault-tolerance
- Trade-off between system reliability and real-time learning explored
- Successful implementation of Monolith in the BytePlus Recommend product demonstrated
- Incremental on-the-fly periodic parameter synchronization mechanism incorporated to scale up online training based on business needs
- Optimization of parameter updates based on model characteristics considering the dominance of sparse parameters in recommendation models

## Summary

The paper discusses the importance of building a scalable and real-time recommendation system for businesses that rely on time-sensitive customer feedback, such as short-videos ranking or online ads. While deep learning frameworks like TensorFlow or PyTorch are widely used, they fall short in recommendation scenarios due to static parameters and dense computations that don't work well with dynamic and sparse features. To address these issues, the authors present Monolith, a system specifically designed for online training. The design of Monolith is driven by observations of application workloads and production environments which sets it apart from other recommendation systems. The contributions of Monolith include a collisionless embedding table with optimizations like expirable embeddings and frequency filtering to reduce memory footprint. It also provides a production-ready online training architecture with high fault-tolerance. The paper further explores how system reliability can be traded-off for real-time learning. The authors demonstrate the successful implementation of Monolith in the BytePlus Recommend product. In order to scale up online training to match business needs, Monolith incorporates an incremental on-the-fly periodic parameter synchronization mechanism. This mechanism takes into account the dominance of sparse parameters in recommendation models and optimizes parameter updates based on model characteristics. Overall, the paper provides insights into the challenges faced by traditional deep learning frameworks in recommendation scenarios and presents a novel approach with Monolith that addresses these challenges effectively.