PackVFL: Efficient HE Packing for Vertical Federated Learning
As an essential tool of secure distributed machine learning, vertical federated learning (VFL) based on homomorphic encryption (HE) suffers from severe efficiency problems due to data inflation and time-consuming operations. To this core, we propose PackVFL, an efficient VFL framework based on packed HE (PackedHE), to accelerate the existing HE-based VFL algorithms. PackVFL packs multiple cleartexts into one ciphertext and supports single-instruction-multiple-data (SIMD)-style parallelism. We focus on designing a high-performant matrix multiplication (MatMult) method since it takes up most of the ciphertext computation time in HE-based VFL. Besides, devising the MatMult method is also challenging for PackedHE because a slight difference in the packing way could predominantly affect its computation and communication costs. Without domain-specific design, directly applying SOTA MatMult methods is hard to achieve optimal. Therefore, we make a three-fold design: 1) we systematically explore the current design space of MatMult and quantify the complexity of existing approaches to provide guidance; 2) we propose a hybrid MatMult method according to the unique characteristics of VFL; 3) we adaptively apply our hybrid method in representative VFL algorithms, leveraging distinctive algorithmic properties to further improve efficiency. As the batch size, feature dimension and model size of VFL scale up to large sizes, PackVFL consistently delivers enhanced performance. Empirically, PackVFL propels existing VFL algorithms to new heights, achieving up to a 51.52X end-to-end speedup. This represents a substantial 34.51X greater speedup compared to the direct application of SOTA MatMult methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningVertical Federated LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Vertical Federated Learning Framework for Horizontally Partitioned Labels
Vertical federated learning is a collaborative machine learning framework to train deep leaning models on vertically partitioned data with privacy-preservation. It attracts much attention both from academia and industry.…
Federated LearningVertical Federated LearningPractical Vertical Federated Learning with Unsupervised Representation Learning
As societal concerns on data privacy recently increase, we have witnessed data silos among multiple parties in various applications. Federated learning emerges as a new learning paradigm that enables multiple parties to …
Federated LearningPrivacy PreservingRepresentation LearningVertical Federated LearningPrivacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
The privacy-preserving federated learning for vertically partitioned data has shown promising results as the solution of the emerging multi-party joint modeling application, in which the data holders (such as government …
Federated LearningPrivacy PreservingPyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN
We introduce PyVertical, a framework supporting vertical federated learning using split neural networks. The proposed framework allows a data scientist to train neural networks on data features vertically partitioned acr…
Federated LearningVertical Federated LearningA Survey on Vertical Federated Learning: From a Layered Perspective
Vertical federated learning (VFL) is a promising category of federated learning for the scenario where data is vertically partitioned and distributed among parties. VFL enriches the description of samples using features …
Federated LearningPrivacy PreservingSurveyVertical Federated Learning