PMGDA: A Preference-based Multiple Gradient Descent Algorithm
It is desirable in many multi-objective machine learning applications, such as multi-task learning with conflicting objectives and multi-objective reinforcement learning, to find a Pareto solution that can match a given preference of a decision maker. These problems are often large-scale with available gradient information but cannot be handled very well by the existing algorithms. To tackle this critical issue, this paper proposes a novel predict-and-correct framework for locating a Pareto solution that fits the preference of a decision maker. In the proposed framework, a constraint function is introduced in the search progress to align the solution with a user-specific preference, which can be optimized simultaneously with multiple objective functions. Experimental results show that our proposed method can efficiently find a particular Pareto solution under the demand of a decision maker for standard multiobjective benchmark, multi-task learning, and multi-objective reinforcement learning problems with more than thousands of decision variables. Code is available at: https://github.com/xzhang2523/pmgda. Our code is current provided in the pgmda.rar attached file and will be open-sourced after publication.}
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Objective Reinforcement LearningMulti-Task Learningreinforcement-learningReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multi-Task Learning with User Preferences: Gradient Descent with Controlled Ascent in Pareto Optimization
Multi-Task Learning (MTL) is a well established learning paradigm for jointly learning models for multiple correlated tasks. Often the tasks conflict requiring trade-offs between them during optimization. Recent advances…
Multi-Task LearningProfiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO). In this work, we propose a novel gradient-based algorithm for profiling Pareto front by …
DiversityProfiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO). In this work, we propose a novel gradient-based algorithm for profiling Pareto front by …
DiversityMulti-Preference Actor Critic
Policy gradient algorithms typically combine discounted future rewards with an estimated value function, to compute the direction and magnitude of parameter updates. However, for most Reinforcement Learning tasks, humans…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Pareto Policy Adaptation
We present a policy gradient method for Multi-Objective Reinforcement Learning under unknown, linear preferences. By enforcing Pareto stationarity, a first-order condition for Pareto optimality, we are able to design a s…
Multi-Objective Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)