Sheaf HyperNetworks for Personalized Federated Learning
Graph hypernetworks (GHNs), constructed by combining graph neural networks (GNNs) with hypernetworks (HNs), leverage relational data across various domains such as neural architecture search, molecular property prediction and federated learning. Despite GNNs and HNs being individually successful, we show that GHNs present problems compromising their performance, such as over-smoothing and heterophily. Moreover, we cannot apply GHNs directly to personalized federated learning (PFL) scenarios, where a priori client relation graph may be absent, private, or inaccessible. To mitigate these limitations in the context of PFL, we propose a novel class of HNs, sheaf hypernetworks (SHNs), which combine cellular sheaf theory with HNs to improve parameter sharing for PFL. We thoroughly evaluate SHNs across diverse PFL tasks, including multi-class classification, traffic and weather forecasting. Additionally, we provide a methodology for constructing client relation graphs in scenarios where such graphs are unavailable. We show that SHNs consistently outperform existing PFL solutions in complex non-IID scenarios. While the baselines' performance fluctuates depending on the task, SHNs show improvements of up to 2.7% in accuracy and 5.3% in lower mean squared error over the best-performing baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningMolecular Property PredictionMulti-class ClassificationNeural Architecture SearchPersonalized Federated LearningProperty PredictionRelationWeather ForecastingSimilar Papers 제목 키워드 기반
FedSheafHN: Personalized Federated Learning on Graph-structured Data
Personalized subgraph Federated Learning (FL) is a task that customizes Graph Neural Networks (GNNs) to individual client needs, accommodating diverse data distributions. However, applying hypernetworks in FL, while aimi…
Federated LearningGraph EmbeddingPersonalized Federated LearningPersonalized Subgraph Federated Learning with Sheaf Collaboration
Graph-structured data is prevalent in many applications. In subgraph federated learning (FL), this data is distributed across clients, each with a local subgraph. Personalized subgraph FL aims to develop a customized mod…
Federated LearningPersonalized Federated Learning using Hypernetworks
Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models in a collaborative way while accounting …
Federated LearningPersonalized Federated LearningPersonalized Federated Hypernetworks for Privacy Preservation in Multi-Task Reinforcement Learning
Multi-Agent Reinforcement Learning currently focuses on implementations where all data and training can be centralized to one machine. But what if local agents are split across multiple tasks, and need to keep data priva…
Federated LearningMulti-agent Reinforcement LearningPersonalized Federated Learningreinforcement-learning+2Hypernetworks for Model-Heterogeneous Personalized Federated Learning
Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning st…
Personalized Federated Learning