Hedging with Sparse Reward Reinforcement Learning
Derivatives, as a critical class of financial instruments, isolate and trade the price attributes of risk assets such as stocks, commodities, and indices, aiding risk management and enhancing market efficiency. However, traditional hedging models, constrained by assumptions such as continuous trading and zero transaction costs, fail to satisfy risk control requirements in complex and uncertain real-world markets. With advances in computing technology and deep learning, data-driven trading strategies are becoming increasingly prevalent. This thesis proposes a derivatives hedging framework integrating deep learning and reinforcement learning. The framework comprises a probabilistic forecasting model and a hedging agent, enabling market probability prediction, derivative pricing, and hedging. Specifically, we design a spatiotemporal attention-based probabilistic financial time series forecasting Transformer to address the scarcity of derivatives hedging data. A low-rank attention mechanism compresses high-dimensional assets into a low-dimensional latent space, capturing nonlinear asset relationships. The Transformer models sequential dependencies within this latent space, improving market probability forecasts and constructing an online training environment for downstream hedging tasks. Additionally, we incorporate generalized geometric Brownian motion to develop a risk-neutral pricing approach for derivatives. We model derivatives hedging as a reinforcement learning problem with sparse rewards and propose a behavior cloning-based recurrent proximal policy optimization (BC-RPPO) algorithm. This pretraining-finetuning framework significantly enhances the hedging agent's performance. Numerical experiments in the U.S. and Chinese financial markets demonstrate our method's superiority over traditional approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningTime Series ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning
This paper proposes a two-phase deep reinforcement learning approach, for hedging variable annuity contracts with both GMMB and GMDB riders, which can address model miscalibration in Black-Scholes financial and constant …
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Deep Reinforcement Learning Algorithms for Option Hedging
Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms h…
Deep Reinforcement LearningQ-Learningreinforcement-learningReinforcement Learning+1Semi-Static and Sparse Variance-Optimal Hedging
We consider hedging of a contingent claim by a 'semi-static' strategy composed of a dynamic position in one asset and static (buy-and-hold) positions in other assets. We give general representations of the optimal strate…
PositionA Risk Sensitive Contract-unified Reinforcement Learning Approach for Option Hedging
We propose a new risk sensitive reinforcement learning approach for the dynamic hedging of options. The approach focuses on the minimization of the tail risk of the final P&L of the seller of an option. Different from mo…
reinforcement-learningReinforcement LearningFeature Hedging: Correlated Features Break Narrow Sparse Autoencoders
It is assumed that sparse autoencoders (SAEs) decompose polysemantic activations into interpretable linear directions, as long as the activations are composed of sparse linear combinations of underlying features. However…