Efficient Distributed Learning with Sparsity
We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines. Computationally, at each round our method only requires the master machine to solve a shifted ell_1 regularized M-estimation problem, and other workers to compute the gradient. In respect of communication, the proposed approach provably matches the estimation error bound of centralized methods within constant rounds of communications (ignoring logarithmic factors). We conduct extensive experiments on both simulated and real world datasets, and demonstrate encouraging performances on high-dimensional regression and classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationregressionSparse LearningSimilar Papers 제목 키워드 기반
L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework
Despite the importance of sparsity in many large-scale applications, there are few methods for distributed optimization of sparsity-inducing objectives. In this paper, we present a communication-efficient framework for L…
Distributed OptimizationEstimation Network Design framework for efficient distributed optimization
Distributed decision problems features a group of agents that can only communicate over a peer-to-peer network, without a central memory. In applications such as network control and data ranking, each agent is only affec…
Distributed OptimizationOptimal Sparsity-Sensitive Bounds for Distributed Mean Estimation
We consider the problem of estimating the mean of a set of vectors, which are stored in a distributed system. This is a fundamental task with applications in distributed SGD and many other distributed problems, where com…
Empowering Distributed Training with Sparsity-driven Data Synchronization
Distributed training is the de facto standard to scale up the training of deep learning models with multiple GPUs. Its performance bottleneck lies in communications for gradient synchronization. Although high tensor spar…
Federated Structured Sparse PCA for Anomaly Detection in IoT Networks
Although federated learning has gained prominence as a privacy-preserving framework tailored for distributed Internet of Things (IoT) environments, current federated principal component analysis (PCA) methods lack integr…
Anomaly DetectionFederated LearningPrivacy Preserving