TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution

2021-02-16 · Karush Suri, Xiao Qi Shi, Konstantinos Plataniotis, Yuri Lawryshyn

Advances in Reinforcement Learning (RL) span a wide variety of applications which motivate development in this area. While application tasks serve as suitable benchmarks for real world problems, RL is seldomly used in practical scenarios consisting of abrupt dynamics. This allows one to rethink the problem setup in light of practical challenges. We present Trade Execution using Reinforcement Learning (TradeR) which aims to address two such practical challenges of catastrophy and surprise minimization by formulating trading as a real-world hierarchical RL problem. Through this lens, TradeR makes use of hierarchical RL to execute trade bids on high frequency real market experiences comprising of abrupt price variations during the 2019 fiscal year COVID19 stock market crash. The framework utilizes an energy-based scheme in conjunction with surprise value function for estimating and minimizing surprise. In a large-scale study of 35 stock symbols from the S&P500 index, TradeR demonstrates robustness to abrupt price changes and catastrophic losses while maintaining profitable outcomes. We hope that our work serves as a motivating example for application of RL to practical problems.

📄 PDF Abstract BibTeX arXiv:2104.00620

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Execution

2024-10-19 · Zijie Zhao, Roy E. Welsch

Leveraging Deep Reinforcement Learning (DRL) in automated stock trading has shown promising results, yet its application faces significant challenges, including the curse of dimensionality, inertia in trading actions, an…

Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

When AI Trading Agents Compete: Adverse Selection of Meta-Orders by Reinforcement Learning-Based Market Making

2025-10-31 · Ali Raza Jafree, Konark Jain, Nick Firoozye arxiv

We investigate the mechanisms by which medium-frequency trading agents are adversely selected by opportunistic high-frequency traders. We use reinforcement learning (RL) within a Hawkes Limit Order Book (LOB) model in or…

Reinforcement Learning

Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimization

2022-12-11 · XiaoDong Li, Pangjing Wu, Chenxin Zou, Qing Li

Designing an intelligent volume-weighted average price (VWAP) strategy is a critical concern for brokers, since traditional rule-based strategies are relatively static that cannot achieve a lower transaction cost in a dy…

Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

On Market Design and Latency Arbitrage

2021-12-26 · Wolfgang Kuhle

We argue that contemporary stock market designs are, due to traders' inability to fully express their preferences over the execution times of their orders, prone to latency arbitrage. In turn, we propose a new order type…

Trade execution games in a Markovian environment

2024-05-12 · Masamitsu Ohnishi, Makoto Shimoshimizu

This paper examines a trade execution game for two large traders in a generalized price impact model. We incorporate a stochastic and sequentially dependent factor that exogenously affects the market price into financial…