Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data
We propose Compressed Vertical Federated Learning (C-VFL) for communication-efficient training on vertically partitioned data. In C-VFL, a server and multiple parties collaboratively train a model on their respective features utilizing several local iterations and sharing compressed intermediate results periodically. Our work provides the first theoretical analysis of the effect message compression has on distributed training over vertically partitioned data. We prove convergence of non-convex objectives at a rate of $O(\frac{1}{\sqrt{T}})$ when the compression error is bounded over the course of training. We provide specific requirements for convergence with common compression techniques, such as quantization and top-$k$ sparsification. Finally, we experimentally show compression can reduce communication by over $90\%$ without a significant decrease in accuracy over VFL without compression.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningQuantizationVertical Federated LearningSimilar Papers 제목 키워드 기반
Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
The privacy-preserving federated learning for vertically partitioned data has shown promising results as the solution of the emerging multi-party joint modeling application, in which the data holders (such as government …
Federated LearningPrivacy PreservingVertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data
Despite enormous research interest and rapid application of federated learning (FL) to various areas, existing studies mostly focus on supervised federated learning under the horizontally partitioned local dataset settin…
Dimensionality ReductionFederated LearningPrivacy-preserving Data Sharing on Vertically Partitioned Data
In this work, we introduce a differentially private method for generating synthetic data from vertically partitioned data, \emph{i.e.}, where data of the same individuals is distributed across multiple data holders or pa…
Privacy PreservingVariational InferencePrivacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent
Combining data from varied sources has considerable potential for knowledge discovery: collaborating data parties can mine data in an expanded feature space, allowing them to explore a larger range of scientific question…
Privacy PreservingAchieving Differential Privacy in Vertically Partitioned Multiparty Learning
Preserving differential privacy has been well studied under centralized setting. However, it's very challenging to preserve differential privacy under multiparty setting, especially for the vertically partitioned case. I…
Privacy Preserving