Feature-distributed sparse regression: a screen-and-clean approach
Most existing approaches to distributed sparse regression assume the data is partitioned by samples. However, for high-dimensional data (D >> N), it is more natural to partition the data by features. We propose an algorithm to distributed sparse regression when the data is partitioned by features rather than samples. Our approach allows the user to tailor our general method to various distributed computing platforms by trading-off the total amount of data (in bits) sent over the communication network and the number of rounds of communication. We show that an implementation of our approach is capable of solving L1-regularized L2 regression problems with millions of features in minutes.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed ComputingregressionSimilar Papers 제목 키워드 기반
A Safe Screening Rule for Sparse Logistic Regression
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implem…
feature selectionregressionOptimality of Graphlet Screening in High Dimensional Variable Selection
Consider a linear regression model where the design matrix X has n rows and p columns. We assume (a) p is much large than n, (b) the coefficient vector beta is sparse in the sense that only a small fraction of its coordi…
Variable SelectionVocal Bursts Intensity PredictionPre-Intervention Prediction of Sparse Autoencoder Steering Side Effects
Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features. We i…
Differentially Private Iterative Screening Rules for Linear Regression
Linear $L_1$-regularized models have remained one of the simplest and most effective tools in data science. Over the past decade, screening rules have risen in popularity as a way to eliminate features when producing the…
Privacy PreservingregressionSafe Screening for Logistic Regression with $\ell_0$-$\ell_2$ Regularization
In logistic regression, it is often desirable to utilize regularization to promote sparse solutions, particularly for problems with a large number of features compared to available labels. In this paper, we present scree…
regression