Robust Reinforcement Learning with Distributional Risk-averse formulation
Robust Reinforcement Learning tries to make predictions more robust to changes in the dynamics or rewards of the system. This problem is particularly important when the dynamics and rewards of the environment are estimated from the data. In this paper, we approximate the Robust Reinforcement Learning constrained with a $\Phi$-divergence using an approximate Risk-Averse formulation. We show that the classical Reinforcement Learning formulation can be robustified using standard deviation penalization of the objective. Two algorithms based on Distributional Reinforcement Learning, one for discrete and one for continuous action spaces are proposed and tested in a classical Gym environment to demonstrate the robustness of the algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributional Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learning
Many real-world domains require safe decision making in uncertain environments. In this work, we introduce a deep reinforcement learning framework for approaching this important problem. We consider a distribution over t…
Improving Robustness via Risk Averse Distributional Reinforcement Learning
One major obstacle that precludes the success of reinforcement learning in real-world applications is the lack of robustness, either to model uncertainties or external disturbances, of the trained policies. Robustness is…
Distributional Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Distributional Method for Risk Averse Reinforcement Learning
We introduce a distributional method for learning the optimal policy in risk averse Markov decision process with finite state action spaces, latent costs, and stationary dynamics. We assume sequential observations of sta…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-averse policies for natural gas futures trading using distributional reinforcement learning
Financial markets have experienced significant instabilities in recent years, creating unique challenges for trading and increasing interest in risk-averse strategies. Distributional Reinforcement Learning (RL) algorithm…
Distributional Reinforcement Learningenergy tradingquantile regressionReinforcement Learning (RL)Risk-Averse Offline Reinforcement Learning
Training Reinforcement Learning (RL) agents in high-stakes applications might be too prohibitive due to the risk associated to exploration. Thus, the agent can only use data previously collected by safe policies. While p…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)