ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems
The integration of Federated Learning (FL) and Multi-Task Learning (MTL) has been explored to address client heterogeneity, with Federated Multi-Task Learning (FMTL) treating each client as a distinct task. However, most existing research focuses on data heterogeneity (e.g., addressing non-IID data) rather than task heterogeneity, where clients solve fundamentally different tasks. Additionally, much of the work relies on centralized settings with a server managing the federation, leaving the more challenging domain of decentralized FMTL largely unexplored. Thus, this work bridges this gap by proposing ColNet, a framework designed for heterogeneous tasks in decentralized federated environments. ColNet divides models into the backbone and task-specific layers, forming groups of similar clients, with group leaders performing conflict-averse cross-group aggregation. A pool of experiments with different federations demonstrated ColNet outperforms the compared aggregation schemes in decentralized settings with label and task heterogeneity scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningMulti-Task LearningSimilar Papers 제목 키워드 기반
Communication-Efficient Federated Non-Linear Bandit Optimization
Federated optimization studies the problem of collaborative function optimization among multiple clients (e.g. mobile devices or organizations) under the coordination of a central server. Since the data is collected sepa…
Distributionally Robust Federated Learning: An ADMM Algorithm
Federated learning (FL) aims to train machine learning (ML) models collaboratively using decentralized data, bypassing the need for centralized data aggregation. Standard FL models often assume that all data come from th…
Federated LearningBayesian Variational Federated Learning and Unlearning in Decentralized Networks
Federated Bayesian learning offers a principled framework for the definition of collaborative training algorithms that are able to quantify epistemic uncertainty and to produce trustworthy decisions. Upon the completion …
Federated LearningVariational InferenceFCOM: A Federated Collaborative Online Monitoring Framework via Representation Learning
Online learning has demonstrated notable potential to dynamically allocate limited resources to monitor a large population of processes, effectively balancing the exploitation of processes yielding high rewards, and the …
Representation LearningCoDream: Exchanging dreams instead of models for federated aggregation with heterogeneous models
Federated Learning (FL) enables collaborative optimization of machine learning models across decentralized data by aggregating model parameters. Our approach extends this concept by aggregating "knowledge" derived from m…
Federated Learning