Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets

2023-01-20 · Peer Nagy, Jan-Peter Calliess, Stefan Zohren

We employ deep reinforcement learning (RL) to train an agent to successfully translate a high-frequency trading signal into a trading strategy that places individual limit orders. Based on the ABIDES limit order book simulator, we build a reinforcement learning OpenAI gym environment and utilise it to simulate a realistic trading environment for NASDAQ equities based on historic order book messages. To train a trading agent that learns to maximise its trading return in this environment, we use Deep Duelling Double Q-learning with the APEX (asynchronous prioritised experience replay) architecture. The agent observes the current limit order book state, its recent history, and a short-term directional forecast. To investigate the performance of RL for adaptive trading independently from a concrete forecasting algorithm, we study the performance of our approach utilising synthetic alpha signals obtained by perturbing forward-looking returns with varying levels of noise. Here, we find that the RL agent learns an effective trading strategy for inventory management and order placing that outperforms a heuristic benchmark trading strategy having access to the same signal.

📄 PDF Abstract BibTeX arXiv:2301.08688

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningManagement OpenAI GymQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Double Q-learning Double Q-learning is an off-policy reinforcement learning algorithm that utilises double estimation to counteract overestimation problems with traditional Q-learning. The…

Double Deep Q-Learning for Optimal Execution

2018-12-17 · Brian Ning, Franco Ho Ting Lin, Sebastian Jaimungal

Optimal trade execution is an important problem faced by essentially all traders. Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, …

Q-Learning

Multi-Agent Reinforcement Learning in a Realistic Limit Order Book Market Simulation

2020-06-10 · Michaël Karpe, Jin Fang, Zhongyao Ma, Chen Wang

Optimal order execution is widely studied by industry practitioners and academic researchers because it determines the profitability of investment decisions and high-level trading strategies, particularly those involving…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning (RL)

Application of deep reinforcement learning for Indian stock trading automation

2021-05-18 · Supriya Bajpai

In stock trading, feature extraction and trading strategy design are the two important tasks to achieve long-term benefits using machine learning techniques. Several methods have been proposed to design trading strategy …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Modelling resource allocation in uncertain system environment through deep reinforcement learning

2021-06-17 · Neel Gandhi, Shakti Mishra

Reinforcement Learning has applications in field of mechatronics, robotics, and other resource-constrained control system. Problem of resource allocation is primarily solved using traditional predefined techniques and mo…