# Safe-FinRL: A Low Bias and Variance Deep Reinforcement Learning Implementation for High-Freq Stock Trading

**Authors:** Zitao Song, Xuyang Jin, Chenliang Li  
**arXiv:** [2206.05910](https://arxiv.org/abs/2206.05910)  
**Date:** 2022-06-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.05910v1/

## Key points

- Growing interest in leveraging Deep Reinforcement Learning (DRL) for quantitative trading strategies
- Introduction of Safe-FinRL, a DRL-based approach tailored for high-frequency stock trading
- Innovation of Safe-FinRL in operating within near-stationary financial environments and minimizing bias and variance in value estimation
- Two-pronged strategy to enhance Safe-FinRL's performance: segmentation of long financial time series data into shorter near-stationary environments and integration of Trace-SAC into the model architecture
- Effectiveness of Safe-FinRL demonstrated through experiments on cryptocurrency market data, showing stable value estimations, consistent policy improvements, and reduction of bias and variance issues

## Summary

, , , , 


In recent years, there has been a growing interest among practitioners in <kw>quantitative finance</kw> to leverage <kw>Deep Reinforcement Learning (DRL)</kw> techniques for developing more effective <kw>quantitative trading (QT)</kw> strategies. However, many existing studies in this field have encountered significant challenges, including the dynamic and non-stationary nature of financial markets. To address these challenges, a team of researchers introduced Safe-FinRL, a novel DRL-based approach tailored specifically for <kw>high-frequency stock trading.</kw>

The key innovation of Safe-FinRL lies in its ability to operate within a near-stationary financial environment while minimizing bias and variance in value estimation. The researchers proposed a two-pronged strategy to enhance the performance of Safe-FinRL. Firstly, they segmented long financial time series data into shorter near-stationary environments to better capture market dynamics. Secondly, they integrated Trace-SAC (Soft Actor-Critic with trace regularization) into the model architecture by incorporating a general retrace operator. This integration aimed to improve policy learning and stabilize value estimation within the near-stationary financial setting. Extensive experiments conducted on cryptocurrency market data demonstrated the effectiveness of Safe-FinRL in providing stable value estimations and consistent policy improvements. Moreover, the approach significantly reduced bias and variance issues typically associated with DRL applications in dynamic financial markets. Overall, Safe-FinRL represents a promising advancement in utilizing DRL for high-frequency stock trading by addressing critical challenges related to market dynamics and bias-variance trade-offs. The research findings underscore the potential of Safe-FinRL to enhance quantitative trading strategies through its innovative design tailored for near-stationary financial environments.