AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization
Vertical federated learning (VFL) is an effective paradigm of training the emerging cross-organizational (e.g., different corporations, companies and organizations) collaborative learning with privacy preserving. Stochastic gradient descent (SGD) methods are the popular choices for training VFL models because of the low per-iteration computation. However, existing SGD-based VFL algorithms are communication-expensive due to a large number of communication rounds. Meanwhile, most existing VFL algorithms use synchronous computation which seriously hamper the computation resource utilization in real-world applications. To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i.e. AsySQN-SGD, -SVRG and -SAGA, are proposed. The proposed AsySQN-type algorithms making descent steps scaled by approximate (without calculating the inverse Hessian matrix explicitly) Hessian information convergence much faster than SGD-based methods in practice and thus can dramatically reduce the number of communication rounds. Moreover, the adopted asynchronous computation can make better use of the computation resource. We theoretically prove the convergence rates of our proposed algorithms for strongly convex problems. Extensive numerical experiments on real-word datasets demonstrate the lower communication costs and better computation resource utilization of our algorithms compared with state-of-the-art VFL algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningPrivacy PreservingVertical Federated LearningSimilar Papers 제목 키워드 기반
An Efficient and Robust System for Vertically Federated Random Forest
As there is a growing interest in utilizing data across multiple resources to build better machine learning models, many vertically federated learning algorithms have been proposed to preserve the data privacy of the par…
Federated LearningFederated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
In a lot of real-world data mining and machine learning applications, data are provided by multiple providers and each maintains private records of different feature sets about common entities. It is challenging to train…
BIG-bench Machine LearningFederated LearningPrivacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
The privacy-preserving federated learning for vertically partitioned data has shown promising results as the solution of the emerging multi-party joint modeling application, in which the data holders (such as government …
Federated LearningPrivacy PreservingSecureBoost+: Large Scale and High-Performance Vertical Federated Gradient Boosting Decision Tree
Gradient boosting decision tree (GBDT) is an ensemble machine learning algorithm, which is widely used in industry, due to its good performance and easy interpretation. Due to the problem of data isolation and the requir…
Federated LearningPrivacy PreservingVertical Federated LearningVAFL: a Method of Vertical Asynchronous Federated Learning
Horizontal Federated learning (FL) handles multi-client data that share the same set of features, and vertical FL trains a better predictor that combine all the features from different clients. This paper targets solving…
Federated Learning