Multi-Task Learning with User Preferences: Gradient Descent with Controlled Ascent in Pareto Optimization

2020-01-01 · ICML 2020 1 · Debabrata Mahapatra, Vaibhav Rajan

Multi-Task Learning (MTL) is a well established learning paradigm for jointly learning models for multiple correlated tasks. Often the tasks conflict requiring trade-offs between them during optimization. Recent advances in multi-objective optimization based MTL have enabled us to use large-scale deep networks to find one or more Pareto optimal solutions. However, they cannot be used to find exact Pareto optimal solutions satisfying user-specified preferences with respect to task-specific losses, that is not only a common requirement in applications but also a useful way to explore the infinite set of Pareto optimal solutions. We develop the first gradient-based multi-objective MTL algorithm to address this problem. Our unique approach combines multiple gradient descent with carefully controlled ascent, that enables it to trace the Pareto front in a principled manner and makes it robust to initialization. Assuming only differentiability of the task-specific loss functions, we provide theoretical guarantees for convergence. We empirically demonstrate the superiority of our algorithm over state-of-the-art methods.

📄 PDF Abstract BibTeX

Code (1)

dbmptr/EPOSearch 공식 구현 pytorch

Tasks

Multi-Task Learning

Perceptron Collaborative Filtering

2024-06-17 · Arya Chakraborty

While multivariate logistic regression classifiers are a great way of implementing collaborative filtering - a method of making automatic predictions about the interests of a user by collecting preferences or taste infor…

Collaborative Filtering Recommendation Systems

Pareto Policy Adaptation

2021-09-29 · ICLR 2022 4 · Panagiotis Kyriakis, Jyotirmoy Deshmukh, Paul Bogdan

We present a policy gradient method for Multi-Objective Reinforcement Learning under unknown, linear preferences. By enforcing Pareto stationarity, a first-order condition for Pareto optimality, we are able to design a s…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

User-Preference Meets Pareto-Optimality: Multi-Objective Bayesian Optimization with Local Gradient Search

2025-02-10 · Joshua Hang Sai Ip, Ankush Chakrabarty, Ali Mesbah, Diego Romeres

Incorporating user preferences into multi-objective Bayesian optimization (MOBO) allows for personalization of the optimization procedure. Preferences are often abstracted in the form of an unknown utility function, esti…

Bayesian Optimization

Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models

2025-07-02 · Chengao Li, Hanyu Zhang, Yunkun Xu, Hongyan Xue 외 arxiv

Reinforcement Learning from Human Feedback (RLHF) has emerged as a powerful technique for aligning large language models (LLMs) with human preferences. However, effectively aligning LLMs with diverse human preferences re…

Reinforcement Learning

Reward-free Alignment for Conflicting Objectives

2026-02-02 · Peter Chen, Xiaopeng Li, Xi Chen, Tianyi Lin arxiv

Direct alignment methods are increasingly used to align large language models (LLMs) with human preferences. However, many real-world alignment problems involve multiple conflicting objectives, where naive aggregation of…