Robust Reinforcement Learning under Diffusion Models for Data with Jumps
Reinforcement Learning (RL) has proven effective in solving complex decision-making tasks across various domains, but challenges remain in continuous-time settings, particularly when state dynamics are governed by stochastic differential equations (SDEs) with jump components. In this paper, we address this challenge by introducing the Mean-Square Bipower Variation Error (MSBVE) algorithm, which enhances robustness and convergence in scenarios involving significant stochastic noise and jumps. We first revisit the Mean-Square TD Error (MSTDE) algorithm, commonly used in continuous-time RL, and highlight its limitations in handling jumps in state dynamics. The proposed MSBVE algorithm minimizes the mean-square quadratic variation error, offering improved performance over MSTDE in environments characterized by SDEs with jumps. Simulations and formal proofs demonstrate that the MSBVE algorithm reliably estimates the value function in complex settings, surpassing MSTDE's performance when faced with jump processes. These findings underscore the importance of alternative error metrics to improve the resilience and effectiveness of RL algorithms in continuous-time frameworks.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Reinforcement Learning for Jump-Diffusions, with Financial Applications
We study continuous-time reinforcement learning (RL) for stochastic control in which system dynamics are governed by jump-diffusion processes. We formulate an entropy-regularized exploratory control problem with stochast…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)From quantum mechanics to finance: Microfoundations for jumps, spikes and high volatility phases in diffusion price processes
We present an agent behavior based microscopic model that induces jumps, spikes and high volatility phases in the price process of a traded asset. We transfer dynamics of thermally activated jumps of an unexcited/ excite…
Optimal investment and consumption with downside risk constraint in jump-diffusion models
This paper extends the results of the article [C. Kl\"{u}ppelberg and S. M. Pergamenchtchikov. Optimal consumption and investment with bounded downside risk for power utility functions. In Optimality and Risk: {\it Moder…
Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls
We study finite-time horizon continuous-time linear-convex reinforcement learning problems in an episodic setting. In this problem, the unknown linear jump-diffusion process is controlled subject to nonsmooth convex cost…
Reinforcement Learning (RL)Diffusion-Jump GNNs: Homophiliation via Learnable Metric Filters
High-order Graph Neural Networks (HO-GNNs) have been developed to infer consistent latent spaces in the heterophilic regime, where the label distribution is not correlated with the graph structure. However, most of the e…
Node Classification