A Projector-Based Approach to Quantifying Total and Excess Uncertainties for Sketched Linear Regression
Linear regression is a classic method of data analysis. In recent years, sketching -- a method of dimension reduction using random sampling, random projections, or both -- has gained popularity as an effective computational approximation when the number of observations greatly exceeds the number of variables. In this paper, we address the following question: How does sketching affect the statistical properties of the solution and key quantities derived from it? To answer this question, we present a projector-based approach to sketched linear regression that is exact and that requires minimal assumptions on the sketching matrix. Therefore, downstream analyses hold exactly and generally for all sketching schemes. Additionally, a projector-based approach enables derivation of key quantities from classic linear regression that account for the combined model- and algorithm-induced uncertainties. We demonstrate the usefulness of a projector-based approach in quantifying and enabling insight on excess uncertainties and bias-variance decompositions for sketched linear regression. Finally, we demonstrate how the insights from our projector-based analyses can be used to produce practical sketching diagnostics to aid the design of judicious sketching schemes.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionregressionSimilar Papers 제목 키워드 기반
OverSketched Newton: Fast Convex Optimization for Serverless Systems
Motivated by recent developments in serverless systems for large-scale computation as well as improvements in scalable randomized matrix algorithms, we develop OverSketched Newton, a randomized Hessian-based optimization…
Distributed OptimizationCommunication-efficient distributed SGD with Sketching
Large-scale distributed training of neural networks is often limited by network bandwidth, wherein the communication time overwhelms the local computation time. Motivated by the success of sketching methods in sub-linear…
The effect of ambiguity in strategic environments: an experiment
We experimentally study a game in which success requires a sufficient total contribution by members of a group. There are significant uncertainties surrounding the chance and the total effort required for success. A theo…
Evidential Uncertainty Quantification: A Variance-Based Perspective
Uncertainty quantification of deep neural networks has become an active field of research and plays a crucial role in various downstream tasks such as active learning. Recent advances in evidential deep learning shed lig…
Active LearningClassificationDomain Adaptationregression+1Localized sketching for matrix multiplication and ridge regression
We consider sketched approximate matrix multiplication and ridge regression in the novel setting of localized sketching, where at any given point, only part of the data matrix is available. This corresponds to a block di…
regression