LOCO: Distributing Ridge Regression with Random Projections
We propose LOCO, an algorithm for large-scale ridge regression which distributes the features across workers on a cluster. Important dependencies between variables are preserved using structured random projections which are cheap to compute and must only be communicated once. We show that LOCO obtains a solution which is close to the exact ridge regression solution in the fixed design setting. We verify this experimentally in a simulation study as well as an application to climate prediction. Furthermore, we show that LOCO achieves significant speedups compared with a state-of-the-art distributed algorithm on a large-scale regression problem.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionSimilar Papers 제목 키워드 기반
DUAL-LOCO: Distributing Statistical Estimation Using Random Projections
We present DUAL-LOCO, a communication-efficient algorithm for distributed statistical estimation. DUAL-LOCO assumes that the data is distributed according to the features rather than the samples. It requires only a singl…
Distributed OptimizationDiffusion index forecasts under weaker loadings: PCA, ridge regression, and random projections
We study the accuracy of forecasts in the diffusion index forecast model with possibly weak loadings. The default option to construct forecasts is to estimate the factors through principal component analysis (PCA) on the…
regressionHigh-dimensional analysis of double descent for linear regression with random projections
We consider linear regression problems with a varying number of random projections, where we provably exhibit a double descent curve for a fixed prediction problem, with a high-dimensional analysis based on random matrix…
regressionMathematical Theory of Collinearity Effects on Machine Learning Variable Importance Measures
In many machine learning problems, understanding variable importance is a central concern. Two common approaches are Permute-and-Predict (PaP), which randomly permutes a feature in a validation set, and Leave-One-Covaria…
ParK: Sound and Efficient Kernel Ridge Regression by Feature Space Partitions
We introduce ParK, a new large-scale solver for kernel ridge regression. Our approach combines partitioning with random projections and iterative optimization to reduce space and time complexity while provably maintainin…
regression