# Reinforcement Learning for Speculative Trading under Exploratory Framework

**Authors:** Yun Zhao, Alex S. L. Tse, Harry Zheng  
**arXiv:** [2604.02035](https://arxiv.org/abs/2604.02035)  
**Date:** 2026-04-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2604.02035v1/

## Key points

- Exploration of speculative trading within the exploratory reinforcement learning (RL) framework proposed by Wang et al. [2020]
- Focus on dynamic optimization problem involving sequential optimal stopping over entry and exit times
- Examination of a relaxed version of the problem using Cox processes controlled by bounded intensities for stopping times
- Characterization of agent's control through a probability measure over jump intensities under exploratory RL formulation
- Derivation of system of exploratory Hamilton-Jacobi-Bellman (HJB) equations and Gibbs distributions as optimal policy
- Establishment of error estimates and demonstration of convergence to value function
- Development of an RL algorithm tailored for speculative trading applications
- Implementation in pairs-trading scenario to showcase practical application

## Summary

In this paper, we explore the realm of speculative trading within the exploratory reinforcement learning (RL) framework proposed by Wang et al. [2020]. Our focus is on a dynamic optimization problem involving sequential optimal stopping over entry and exit times, considering general utility functions and price processes. To tackle this complex issue, we first examine a relaxed version of the problem where stopping times are represented by the jump times of Cox processes controlled by bounded intensities. Under the exploratory RL formulation, we characterize the agent's control through a probability measure over jump intensities while regularizing their objective function with Shannon's differential entropy. This unique approach leads us to derive a system of exploratory Hamilton-Jacobi-Bellman (HJB) equations and Gibbs distributions in closed-form as the optimal policy. We also establish error estimates and demonstrate convergence of the RL objective to the value function of the original problem. Furthermore, our contribution extends to developing an RL algorithm tailored for speculative trading applications. By showcasing its implementation in a pairs-trading scenario, we illustrate how our theoretical framework can be effectively put into practice. Overall, this work adds to the continuous-time RL literature by addressing sequential optimal stopping problems under general diffusion dynamics and utility functions while emphasizing exploration in decision-making processes.