Multi-Task Learning via Task Multi-Clustering
Multi-task learning has the potential to facilitate learning of shared representations between tasks, leading to better task performance. Some sets of tasks are related, and can share many features that are useful latent representations for these tasks. Other sets of tasks are less related, possibly sharing some features, but also competing on the representational resources of shared parameters. We propose to discover how to share parameters between related tasks and split parameters between conflicting tasks, by learning a multi-clustering of the tasks. We present a mixture-of-experts model, where each cluster is an expert that extracts a feature vector from the input, and each task belongs to a set of clusters whose experts it can mix. In experiments on the CIFAR-100 MTL domain, multi-clustering outperforms a model that mixes all experts in accuracy and computation time. The results suggest that the performance of our method is robust to regularization that increases the model's sparsity when sufficient data is available, and can benefit from sparser models as data becomes scarcer.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringMixture-of-ExpertsMulti-Task LearningSimilar Papers 제목 키워드 기반
Convex Discriminative Multitask Clustering
Multitask clustering tries to improve the clustering performance of multiple tasks simultaneously by taking their relationship into account. Most existing multitask clustering algorithms fall into the type of generative …
ClusteringSemisoft Task Clustering for Multi-Task Learning
Multi-task learning (MTL) aims to improve the performance of multiple related prediction tasks by leveraging useful information from them. Due to their flexibility and ability to reduce unknown coefficients substantially…
ClusteringMulti-Task LearningMulti-task learning via robust regularized clustering with non-convex group penalties
Multi-task learning (MTL) aims to improve estimation and prediction performance by sharing common information among related tasks. One natural assumption in MTL is that tasks are classified into clusters based on their c…
ClusteringMulti-Task LearningGCFAgg: Global and Cross-view Feature Aggregation for Multi-view Clustering
Multi-view clustering can partition data samples into their categories by learning a consensus representation in unsupervised way and has received more and more attention in recent years. However, most existing deep clus…
ClusteringContrastive LearningDeep ClusteringRobust multi-task boosting using clustering and local ensembling
Multi-Task Learning (MTL) aims to boost predictive performance by sharing information across related tasks, yet conventional methods often suffer from negative transfer when unrelated or noisy tasks are forced to share r…
Multi-Task Learning