Communication-efficient Distributed Sparse Linear Discriminant Analysis
We propose a communication-efficient distributed estimation method for sparse linear discriminant analysis (LDA) in the high dimensional regime. Our method distributes the data of size $N$ into $m$ machines, and estimates a local sparse LDA estimator on each machine using the data subset of size $N/m$. After the distributed estimation, our method aggregates the debiased local estimators from $m$ machines, and sparsifies the aggregated estimator. We show that the aggregated estimator attains the same statistical rate as the centralized estimation method, as long as the number of machines $m$ is chosen appropriately. Moreover, we prove that our method can attain the model selection consistency under a milder condition than the centralized method. Experiments on both synthetic and real datasets corroborate our theory.
Code (0)
등록된 구현이 없습니다.
Tasks
Model SelectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distributed Sparse Multicategory Discriminant Analysis
This paper proposes a convex formulation for sparse multicategory linear discriminant analysis and then extend it to the distributed setting when data are stored across multiple sites. The key observation is that for the…
Kullback-Leibler Penalized Sparse Discriminant Analysis for Event-Related Potential Classification
A brain computer interface (BCI) is a system which provides direct communication between the mind of a person and the outside world by using only brain activity (EEG). The event-related potential (ERP)-based BCI problem …
Brain Computer InterfaceEEGElectroencephalogram (EEG)ERP+2A State-Space Approach to Nonstationary Discriminant Analysis
Classical discriminant analysis assumes identically distributed training data, yet in many applications observations are collected over time and the class-conditional distributions drift. This population drift renders st…
Alternating direction method of multipliers for penalized zero-variance discriminant analysis
We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuris…
feature selectionGeneral ClassificationTime SeriesTime Series Analysis+1A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication
The large communication overhead has imposed a bottleneck on the performance of distributed Stochastic Gradient Descent (SGD) for training deep neural networks. Previous works have demonstrated the potential of using gr…
Quantization