# Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools

**Authors:** Wentao Zhang  
**arXiv:** [2311.10801](https://arxiv.org/abs/2311.10801)  
**Date:** 2023-11-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.10801v1/

## Key points

- Portfolio management (PM) involves reallocating capital into different stocks to maximize long-term profits
- Reinforcement learning (RL) is a promising approach for training profitable agents for PM by interacting with financial markets
- Existing RL methods have focused on fixed stock pools, not customizable stock pools (CSPs)
- EarnMore framework utilizes Maskable Stock Representation to handle PM with CSPs through one-shot training in a global stock pool (GSP)
- EarnMore aims to improve performance and efficiency in PM with CSPs by masking out stocks outside the target pool, learning meaningful stock representations, and implementing re-weighting mechanisms
- EarnMore significantly outperformed 14 state-of-the-art baselines in experiments across two real US financial markets, showing over 40% improvement in profit
- EarnMore effectively met investors' preferences and decisions in the trading process by constructing customizable stock pools based on different investor preferences
- Ablation studies were conducted to assess the usefulness of each component of EarnMore and its efficiency in handling PM with CSPs
- EarnMore demonstrated generality and effectiveness across various scenarios and market conditions, presenting a promising approach for optimizing portfolio management strategies

## Summary

Portfolio management (PM) is a critical financial trading task that involves reallocating capital into different stocks to maximize long-term profits. Recently, reinforcement learning (RL) has emerged as a promising approach to train profitable agents for PM by interacting with financial markets. However, existing RL methods have primarily focused on fixed stock pools, which do not align with the practical demands of investors who often have customizable stock pools (CSPs) based on their individual preferences and market conditions. To address this challenge, a new framework called EarnMore has been proposed. This framework utilizes Maskable Stock Representation to handle PM with CSPs through one-shot training in a global stock pool (GSP). By masking out stocks outside the target pool, learning meaningful stock representations through self-supervised processes, and implementing re-weighting mechanisms to concentrate on favorable stocks, EarnMore aims to improve performance and efficiency in PM with CSPs. In a series of experiments conducted using daily k-line data for over 3,000 US stocks from Yahoo Finance, EarnMore was evaluated against 14 state-of-the-art baselines across two real US financial markets. The results demonstrated that EarnMore significantly outperformed the baselines in terms of six popular financial metrics, showing over 40% improvement in profit. Furthermore, the experiments included constructing six customizable stock pools based on different investor preferences in two US financial markets. This showcased how EarnMore effectively met investors' preferences and decisions in the trading process. Ablation studies were also conducted to answer key questions regarding the usefulness of each component of EarnMore, why direct methods for PM with CSPs are ineffective, and the efficiency of the model. Overall, these experiments highlighted the generality and effectiveness of EarnMore in handling PM with CSPs across various scenarios and market conditions. By demonstrating superior returns compared to baseline methods and showcasing adaptability to different investor preferences, EarnMore presents a promising approach for optimizing portfolio management strategies in dynamic financial markets.