Robust Risk-Aware Reinforcement Learning
We present a reinforcement learning (RL) approach for robust optimisation of risk-aware performance criteria. To allow agents to express a wide variety of risk-reward profiles, we assess the value of a policy using rank dependent expected utility (RDEU). RDEU allows the agent to seek gains, while simultaneously protecting themselves against downside risk. To robustify optimal policies against model uncertainty, we assess a policy not by its distribution, but rather, by the worst possible distribution that lies within a Wasserstein ball around it. Thus, our problem formulation may be viewed as an actor/agent choosing a policy (the outer problem), and the adversary then acting to worsen the performance of that strategy (the inner problem). We develop explicit policy gradient formulae for the inner and outer problems, and show its efficacy on three prototypical financial problems: robust portfolio allocation, optimising a benchmark, and statistical arbitrage.
Code (1)
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Risk-Aware Active Inverse Reinforcement Learning
Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, no…
Active Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-Aware Reinforcement Learning for Mobile Manipulation
For robots to successfully transition from lab settings to everyday environments, they must begin to reason about the risks associated with their actions and make informed, risk-aware decisions. This is particularly true…
Reinforcement LearningRisk-Aware Transfer in Reinforcement Learning using Successor Features
Sample efficiency and risk-awareness are central to the development of practical reinforcement learning (RL) for complex decision-making. The former can be addressed by transfer learning and the latter by optimizing some…
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1RA-PbRL: Provably Efficient Risk-Aware Preference-Based Reinforcement Learning
Reinforcement Learning from Human Feedback (RLHF) has recently surged in popularity, particularly for aligning large language models and other AI systems with human intentions. At its core, RLHF can be viewed as a specia…
Autonomous Drivingreinforcement-learningReinforcement LearningRisk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving
Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, t…
Autonomous DrivingMotion PlanningOpenAI Gymreinforcement-learning+1