Federated Transformer: Multi-Party Vertical Federated Learning on Practical Fuzzily Linked Data
Federated Learning (FL) is an evolving paradigm that enables multiple parties to collaboratively train models without sharing raw data. Among its variants, Vertical Federated Learning (VFL) is particularly relevant in real-world, cross-organizational collaborations, where distinct features of a shared instance group are contributed by different parties. In these scenarios, parties are often linked using fuzzy identifiers, leading to a common practice termed as multi-party fuzzy VFL. Existing models generally address either multi-party VFL or fuzzy VFL between two parties. Extending these models to practical multi-party fuzzy VFL typically results in significant performance degradation and increased costs for maintaining privacy. To overcome these limitations, we introduce the Federated Transformer (FeT), a novel framework that supports multi-party VFL with fuzzy identifiers. FeT innovatively encodes these identifiers into data representations and employs a transformer architecture distributed across different parties, incorporating three new techniques to enhance performance. Furthermore, we have developed a multi-party privacy framework for VFL that integrates differential privacy with secure multi-party computation, effectively protecting local representations while minimizing associated utility costs. Our experiments demonstrate that the FeT surpasses the baseline models by up to 46\% in terms of accuracy when scaled to 50 parties. Additionally, in two-party fuzzy VFL settings, FeT also shows improved performance and privacy over cutting-edge VFL models.
Code (1)
Tasks
Entity AlignmentFederated LearningVertical Federated LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interpret Federated Learning with Shapley Values
Federated Learning is introduced to protect privacy by distributing training data into multiple parties. Each party trains its own model and a meta-model is constructed from the sub models. In this way the details of the…
Feature ImportanceFederated LearningVertical Federated LearningPrivacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
The privacy-preserving federated learning for vertically partitioned data has shown promising results as the solution of the emerging multi-party joint modeling application, in which the data holders (such as government …
Federated LearningPrivacy PreservingPractical Vertical Federated Learning with Unsupervised Representation Learning
As societal concerns on data privacy recently increase, we have witnessed data silos among multiple parties in various applications. Federated learning emerges as a new learning paradigm that enables multiple parties to …
Federated LearningPrivacy PreservingRepresentation LearningVertical Federated LearningParallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
Federated Learning is a new distributed learning mechanism which allows model training on a large corpus of decentralized data owned by different data providers, without sharing or leakage of raw data. According to the c…
Federated LearningregressionTransfer LearningVertical Federated LearningVertical Semi-Federated Learning for Efficient Online Advertising
The traditional vertical federated learning schema suffers from two main issues: 1) restricted applicable scope to overlapped samples and 2) high system challenge of real-time federated serving, which limits its applicat…
Federated LearningVertical Federated Learning