# Climber: Toward Efficient Scaling Laws for Large Recommendation Models

**Authors:** Songpei Xu, Shijia Wang, Da Guo, Xianwen Guo, Qiang Xiao, Bin Huang, Guanlin Wu, Chuanjiang Luo  
**arXiv:** [2502.09888](https://arxiv.org/abs/2502.09888)  
**Date:** 2025-02-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.09888v2/

## Key points

- Transformer-based generative models have shown success in recommendation systems
- Challenges with Transformers in recommendation systems include suboptimal scaling, structural incompatibilities, and online inference latency constraints
- Climber framework addresses these challenges with innovative model architecture and acceleration techniques
- Core innovations of Climber include multi-scale sequence extraction and dynamic temperature modulation to enhance efficiency
- Climber achieves a 5.15$\times$ throughput gain without compromising performance quality through batched processing and memory-efficient caching strategies
- Extensive offline experiments validate that Climber exhibits ideal scaling compared to existing models
- Climber drives continuous online metric growth (12.19\% overall lift) without prohibitive resource costs

## Summary

In the realm of recommendation systems, Transformer-based generative models have shown great success in various domains. They have demonstrated diverse scaling law manifestations. However, challenges persist when applying Transformers to recommendation systems. These challenges include suboptimal scaling with increased computational resources due to structural incompatibilities with recommendation-specific features like multi-source data heterogeneity and critical online inference latency constraints that escalate with longer user behavior sequences and heightened computational demands. To address these persistent challenges, a novel and efficient recommendation framework called Climber has been proposed. Climber comprises two synergistic components: an innovative model architecture for efficient scaling and co-designed acceleration techniques. The proposed model introduces two core innovations to enhance efficiency: multi-scale sequence extraction and dynamic temperature modulation. These innovations reduce time complexity by a constant factor and adjust attention distributions to accommodate multi-scenario and multi-behavior patterns. Complemented by acceleration techniques, Climber achieves a remarkable 5.15$\times$ throughput gain without compromising performance quality through the adoption of "single user, multiple item" batched processing and memory-efficient Key-Value caching strategies. Extensive offline experiments conducted on multiple datasets validate that Climber exhibits a more ideal scaling curve compared to existing models. Notably, Climber stands out as the first publicly documented framework where controlled model scaling drives continuous online metric growth (12.19\% overall lift) without incurring prohibitive resource costs. This groundbreaking framework has been successfully deployed on Netease Cloud Music, one of China's largest music streaming platforms, catering to tens of millions of users on a daily basis.