Multi-Task Learning with Multi-Task Optimization
Multi-task learning solves multiple correlated tasks. However, conflicts may exist between them. In such circumstances, a single solution can rarely optimize all the tasks, leading to performance trade-offs. To arrive at a set of optimized yet well-distributed models that collectively embody different trade-offs in one algorithmic pass, this paper proposes to view Pareto multi-task learning through the lens of multi-task optimization. Multi-task learning is first cast as a multi-objective optimization problem, which is then decomposed into a diverse set of unconstrained scalar-valued subproblems. These subproblems are solved jointly using a novel multi-task gradient descent method, whose uniqueness lies in the iterative transfer of model parameters among the subproblems during the course of optimization. A theorem proving faster convergence through the inclusion of such transfers is presented. We investigate the proposed multi-task learning with multi-task optimization for solving various problem settings including image classification, scene understanding, and multi-target regression. Comprehensive experiments confirm that the proposed method significantly advances the state-of-the-art in discovering sets of Pareto-optimized models. Notably, on the large image dataset we tested on, namely NYUv2, the hypervolume convergence achieved by our method was found to be nearly two times faster than the next-best among the state-of-the-art.
Code (0)
등록된 구현이 없습니다.
Tasks
Automated Theorem Provingimage-classificationImage ClassificationMulti-target regressionMulti-Task LearningScene UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multi-Tasking Genetic Algorithm (MTGA) for Fuzzy System Optimization
Multi-task learning uses auxiliary data or knowledge from relevant tasks to facilitate the learning in a new task. Multi-task optimization applies multi-task learning to optimization to study how to effectively and effic…
Multi-Task LearningCan Optimization Trajectories Explain Multi-Task Transfer?
Despite the widespread adoption of multi-task training in deep learning, little is understood about how multi-task learning (MTL) affects generalization. Prior work has conjectured that the negative effects of MTL are du…
Multi-Task LearningMUMBO: MUlti-task Max-value Bayesian Optimization
We propose MUMBO, the first high-performing yet computationally efficient acquisition function for multi-task Bayesian optimization. Here, the challenge is to perform efficient optimization by evaluating low-cost functio…
Bayesian OptimizationSelective Task Group Updates for Multi-Task Optimization
Multi-task learning enables the acquisition of task-generic knowledge by training multiple tasks within a unified architecture. However, training all tasks together in a single architecture can lead to performance degrad…
Multi-Task LearningMulti-Task Bayesian Optimization for Tuning Decentralized Trajectory Generation in Multi-UAV Systems
This paper investigates the use of Multi-Task Bayesian Optimization for tuning decentralized trajectory generation algorithms in multi-drone systems. We treat each task as a trajectory generation scenario defined by a sp…
Gaussian Processes