Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification
Federated Knowledge Graphs Embedding learning (FKGE) encounters challenges in communication efficiency stemming from the considerable size of parameters and extensive communication rounds. However, existing FKGE methods only focus on reducing communication rounds by conducting multiple rounds of local training in each communication round, and ignore reducing the size of parameters transmitted within each communication round. To tackle the problem, we first find that universal reduction in embedding precision across all entities during compression can significantly impede convergence speed, underscoring the importance of maintaining embedding precision. We then propose bidirectional communication-efficient FedS based on Entity-Wise Top-K Sparsification strategy. During upload, clients dynamically identify and upload only the Top-K entity embeddings with the greater changes to the server. During download, the server first performs personalized embedding aggregation for each client. It then identifies and transmits the Top-K aggregated embeddings to each client. Besides, an Intermittent Synchronization Mechanism is used by FedS to mitigate negative effect of embedding inconsistency among shared entities of clients caused by heterogeneity of Federated Knowledge Graph. Extensive experiments across three datasets showcase that FedS significantly enhances communication efficiency with negligible (even no) performance degradation.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity EmbeddingsGraph EmbeddingKnowledge Graph EmbeddingKnowledge GraphsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation
Federated learning (FL) can be essential in knowledge representation, reasoning, and data mining applications over multi-source knowledge graphs (KGs). A recent study FedE first proposes an FL framework that shares entit…
Entity EmbeddingsFederated LearningKnowledge GraphsLink Prediction+2Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation
Federated Knowledge Graph Embedding (FKGE) aims to facilitate collaborative learning of entity and relation embeddings from distributed Knowledge Graphs (KGs) across multiple clients, while preserving data privacy. Train…
Graph EmbeddingKnowledge DistillationKnowledge Graph EmbeddingKnowledge GraphsPersonalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph
Federated Knowledge Graph Embedding (FKGE) has recently garnered considerable interest due to its capacity to extract expressive representations from distributed knowledge graphs, while concurrently safeguarding the priv…
Entity Embeddingsglobal-optimizationGraph EmbeddingKnowledge Graph Embedding+2FedC4: Graph Condensation Meets Client-Client Collaboration for Efficient and Private Federated Graph Learning
Federated Graph Learning (FGL) is an emerging distributed learning paradigm that enables collaborative model training over decentralized graph-structured data while preserving local privacy. Existing FGL methods can be c…
Graph LearningRethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approach
Recent studies of federated graph foundational models (FedGFMs) break the idealized and untenable assumption of having centralized data storage to train graph foundation models, and accommodate the reality of distributed…
Contrastive LearningGeneral Knowledge