Double Deep Q-Learning for Optimal Execution

2018-12-17 · Brian Ning, Franco Ho Ting Lin, Sebastian Jaimungal

Optimal trade execution is an important problem faced by essentially all traders. Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, we instead take a model free approach and develop a variation of Deep Q-Learning to estimate the optimal actions of a trader. The model is a fully connected Neural Network trained using Experience Replay and Double DQN with input features given by the current state of the limit order book, other trading signals, and available execution actions, while the output is the Q-value function estimating the future rewards under an arbitrary action. We apply our model to nine different stocks and find that it outperforms the standard benchmark approach on most stocks using the measures of (i) mean and median out-performance, (ii) probability of out-performance, and (iii) gain-loss ratios.

📄 PDF Abstract BibTeX arXiv:1812.06600

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learning

Methods 이 논문이 사용한 방법론

Double Q-learning Double Q-learning is an off-policy reinforcement learning algorithm that utilises double estimation to counteract overestimation problems with traditional Q-learning. The…

Double DQN 설명 없음

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Multi-Agent Reinforcement Learning in a Realistic Limit Order Book Market Simulation

2020-06-10 · Michaël Karpe, Jin Fang, Zhongyao Ma, Chen Wang

Optimal order execution is widely studied by industry practitioners and academic researchers because it determines the profitability of investment decisions and high-level trading strategies, particularly those involving…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning (RL)

Reinforcement Learning for Optimal Execution when Liquidity is Time-Varying

2024-02-19 · Andrea Macrì, Fabrizio Lillo

Optimal execution is an important problem faced by any trader. Most solutions are based on the assumption of constant market impact, while liquidity is known to be dynamic. Moreover, models with time-varying liquidity ty…

Q-Learning reinforcement-learning Reinforcement Learning

Protocol for Executing and Benchmarking Eight Computational Doublet-Detection Methods in Single-Cell RNA Sequencing Data Analysis

2021-01-21 · Nan Miles Xi, Jingyi Jessica Li

The existence of doublets is a key confounder in single-cell RNA sequencing (scRNA-seq) data analysis. Computational methods have been developed for detecting doublets from scRNA-seq data. We developed an R package Doubl…

Benchmarking

Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution

2025-11-19 · Tomas Espana, Yadh Hafsi, Fabrizio Lillo, Edoardo Vittori arxiv

We investigate the use of Reinforcement Learning for the optimal execution of meta-orders, where the objective is to execute incrementally large orders while minimizing implementation shortfall and market impact over an …

Reinforcement Learning

Optimally Solving Simultaneous-Move Dec-POMDPs: The Sequential Central Planning Approach

2024-08-23 · Johan Peralez, Aurèlien Delage, Jacopo Castellini, Rafael F. Cunha 외

The centralized training for decentralized execution paradigm emerged as the state-of-the-art approach to $\epsilon$-optimally solving decentralized partially observable Markov decision processes. However, scalability re…