Robust Coreset Construction for Distributed Machine Learning
Coreset, which is a summary of the original dataset in the form of a small weighted set in the same sample space, provides a promising approach to enable machine learning over distributed data. Although viewed as a proxy of the original dataset, each coreset is only designed to approximate the cost function of a specific machine learning problem, and thus different coresets are often required to solve different machine learning problems, increasing the communication overhead. We resolve this dilemma by developing robust coreset construction algorithms that can support a variety of machine learning problems. Motivated by empirical evidence that suitably-weighted k-clustering centers provide a robust coreset, we harden the observation by establishing theoretical conditions under which the coreset provides a guaranteed approximation for a broad range of machine learning problems, and developing both centralized and distributed algorithms to generate coresets satisfying the conditions. The robustness of the proposed algorithms is verified through extensive experiments on diverse datasets with respect to both supervised and unsupervised learning problems.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningClusteringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On Coresets for Support Vector Machines
We present an efficient coreset construction algorithm for large-scale Support Vector Machine (SVM) training in Big Data and streaming applications. A coreset is a small, representative subset of the original data points…
Small Data Image ClassificationJoint Coreset Construction and Quantization for Distributed Machine Learning
Coresets are small, weighted summaries of larger datasets, aiming at providing provable error bounds for machine learning (ML) tasks while significantly reducing the communication and computation costs. To achieve a bett…
BIG-bench Machine LearningQuantizationPractical Coreset Constructions for Machine Learning
We investigate coresets - succinct, small summaries of large data sets - so that solutions found on the summary are provably competitive with solution found on the full data set. We provide an overview over the state-of-…
BIG-bench Machine LearningClusteringregressionFedCore: Straggler-Free Federated Learning with Distributed Coresets
Federated learning (FL) is a machine learning paradigm that allows multiple clients to collaboratively train a shared model while keeping their data on-premise. However, the straggler issue, due to slow clients, often hi…
Federated LearningIntroduction to Core-sets: an Updated Survey
In optimization or machine learning problems we are given a set of items, usually points in some metric space, and the goal is to minimize or maximize an objective function over some space of candidate solutions. For exa…
ClusteringData SummarizationSurvey