Multi-Agent Reinforcement Learning in a Realistic Limit Order Book Market Simulation

2020-06-10 · Michaël Karpe, Jin Fang, Zhongyao Ma, Chen Wang

Optimal order execution is widely studied by industry practitioners and academic researchers because it determines the profitability of investment decisions and high-level trading strategies, particularly those involving large volumes of orders. However, complex and unknown market dynamics pose significant challenges for the development and validation of optimal execution strategies. In this paper, we propose a model-free approach by training Reinforcement Learning (RL) agents in a realistic market simulation environment with multiple agents. First, we configure a multi-agent historical order book simulation environment for execution tasks built on an Agent-Based Interactive Discrete Event Simulation (ABIDES) [arXiv:1904.12066]. Second, we formulate the problem of optimal execution in an RL setting where an intelligent agent can make order execution and placement decisions based on market microstructure trading signals in High Frequency Trading (HFT). Third, we develop and train an RL execution agent using the Double Deep Q-Learning (DDQL) algorithm in the ABIDES environment. In some scenarios, our RL agent converges towards a Time-Weighted Average Price (TWAP) strategy. Finally, we evaluate the simulation with our RL agent by comparing it with a market replay simulation using real market Limit Order Book (LOB) data.

📄 PDF Abstract BibTeX arXiv:2006.05574

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Towards Inverse Reinforcement Learning for Limit Order Book Dynamics

2019-06-11 · Jacobo Roa-Vicens, Cyrine Chtourou, Angelos Filos, Francisco Rullan 외

Multi-agent learning is a promising method to simulate aggregate competitive behaviour in finance. Learning expert agents' reward functions through their external demonstrations is hence particularly relevant for subsequ…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets

2023-01-20 · Peer Nagy, Jan-Peter Calliess, Stefan Zohren

We employ deep reinforcement learning (RL) to train an agent to successfully translate a high-frequency trading signal into a trading strategy that places individual limit orders. Based on the ABIDES limit order book sim…

Deep Reinforcement Learning Management OpenAI Gym Q-Learning+3

Optimizing Market Making using Multi-Agent Reinforcement Learning

2018-12-26 · Yagna Patel

In this paper, reinforcement learning is applied to the problem of optimizing market making. A multi-agent reinforcement learning framework is used to optimally place limit orders that lead to successful trades. The fram…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution

2025-11-19 · Tomas Espana, Yadh Hafsi, Fabrizio Lillo, Edoardo Vittori arxiv

We investigate the use of Reinforcement Learning for the optimal execution of meta-orders, where the objective is to execute incrementally large orders while minimizing implementation shortfall and market impact over an …

Reinforcement Learning

Learning Practical Communication Strategies in Cooperative Multi-Agent Reinforcement Learning

2022-09-02 · Diyi Hu, Chi Zhang, Viktor Prasanna, Bhaskar 외

In Multi-Agent Reinforcement Learning, communication is critical to encourage cooperation among agents. Communication in realistic wireless networks can be highly unreliable due to network conditions varying with agents'…