Improving Robustness via Risk Averse Distributional Reinforcement Learning
One major obstacle that precludes the success of reinforcement learning in real-world applications is the lack of robustness, either to model uncertainties or external disturbances, of the trained policies. Robustness is critical when the policies are trained in simulations instead of real world environment. In this work, we propose a risk-aware algorithm to learn robust policies in order to bridge the gap between simulation training and real-world implementation. Our algorithm is based on recently discovered distributional RL framework. We incorporate CVaR risk measure in sample based distributional policy gradients (SDPG) for learning risk-averse policies to achieve robustness against a range of system disturbances. We validate the robustness of risk-aware SDPG on multiple environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributional Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Robust Reinforcement Learning with Distributional Risk-averse formulation
Robust Reinforcement Learning tries to make predictions more robust to changes in the dynamics or rewards of the system. This problem is particularly important when the dynamics and rewards of the environment are estimat…
Distributional Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-Averse Offline Reinforcement Learning
Training Reinforcement Learning (RL) agents in high-stakes applications might be too prohibitive due to the risk associated to exploration. Thus, the agent can only use data previously collected by safe policies. While p…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learning
Many real-world domains require safe decision making in uncertain environments. In this work, we introduce a deep reinforcement learning framework for approaching this important problem. We consider a distribution over t…
Distributional Method for Risk Averse Reinforcement Learning
We introduce a distributional method for learning the optimal policy in risk averse Markov decision process with finite state action spaces, latent costs, and stationary dynamics. We assume sequential observations of sta…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-averse policies for natural gas futures trading using distributional reinforcement learning
Financial markets have experienced significant instabilities in recent years, creating unique challenges for trading and increasing interest in risk-averse strategies. Distributional Reinforcement Learning (RL) algorithm…
Distributional Reinforcement Learningenergy tradingquantile regressionReinforcement Learning (RL)