Thresholded Lexicographic Ordered Multiobjective Reinforcement Learning
Lexicographic multi-objective problems, which impose a lexicographic importance order over the objectives, arise in many real-life scenarios. Existing Reinforcement Learning work directly addressing lexicographic tasks has been scarce. The few proposed approaches were all noted to be heuristics without theoretical guarantees as the Bellman equation is not applicable to them. Additionally, the practical applicability of these prior approaches also suffers from various issues such as not being able to reach the goal state. While some of these issues have been known before, in this work we investigate further shortcomings, and propose fixes for improving practical performance in many cases. We also present a policy optimization approach using our Lexicographic Projection Optimization (LPO) algorithm that has the potential to address these theoretical and practical concerns. Finally, we demonstrate our proposed algorithms on benchmark problems.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Lexicographic Multiarmed Bandit
We consider a multiobjective multiarmed bandit problem with lexicographically ordered objectives. In this problem, the goal of the learner is to select arms that are lexicographic optimal as much as possible without know…
Beyond Scalar Rewards: An Axiomatic Framework for Lexicographic MDPs
Recent work has formalized the reward hypothesis through the lens of expected utility theory, by interpreting reward as utility. Hausner's foundational work showed that dropping the continuity axiom leads to a generaliza…
gTLO: A Generalized and Non-linear Multi-Objective Deep Reinforcement Learning Approach
In real-world decision optimization, often multiple competing objectives must be taken into account. Following classical reinforcement learning, these objectives have to be combined into a single reward function. In cont…
Deep Reinforcement LearningDeep-Sea Treasure, Image versionMulti-Objective Reinforcement Learningreinforcement-learning+2Inferring Lexicographically-Ordered Rewards from Preferences
Modeling the preferences of agents over a set of alternatives is a principal concern in many areas. The dominant approach has been to find a single reward/utility function with the property that alternatives yielding hig…
Navigation In Urban Environments Amongst Pedestrians Using Multi-Objective Deep Reinforcement Learning
Urban autonomous driving in the presence of pedestrians as vulnerable road users is still a challenging and less examined research problem. This work formulates navigation in urban environments as a multi objective reinf…
Autonomous DrivingAutonomous NavigationDeep Reinforcement LearningMulti-Objective Reinforcement Learning+3