paper-with-me

Papers

High-dimensional Ordinary Least-squares Projection for Screening Variables

2015-06-05 · Xiangyu Wang, Chenlei Leng

Variable selection is a challenging issue in statistical applications when the number of predictors $p$ far exceeds the number of observations $n$. In this ultra-high dimensional setting, the sure independence screening (SIS) procedure was introduced to significantly reduce the dimensionality by preserving the true model with overwhelming probability, before a refined second stage analysis. However, the aforementioned sure screening property strongly relies on the assumption that the important variables in the model have large marginal correlations with the response, which rarely holds in reality. To overcome this, we propose a novel and simple screening technique called the high-dimensional ordinary least-squares projection (HOLP). We show that HOLP possesses the sure screening property and gives consistent variable selection without the strong correlation assumption, and has a low computational complexity. A ridge type HOLP procedure is also discussed. Simulation study shows that HOLP performs competitively compared to many other marginal correlation based methods. An application to a mammalian eye disease data illustrates the attractiveness of HOLP.

📄 PDF Abstract BibTeX arXiv:1506.01782

Code (0)

등록된 구현이 없습니다.

Tasks

Variable SelectionVocal Bursts Intensity Prediction

Similar Papers 제목 키워드 기반

A Risk Comparison of Ordinary Least Squares vs Ridge Regression

2011-05-04 · Paramveer S. Dhillon, Dean P. Foster, Sham M. Kakade, Lyle H. Ungar

We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then pe…

regression

Compressed Least-Squares Regression

2009-12-01 · NeurIPS 2009 12 · Odalric Maillard, Rémi Munos

We consider the problem of learning, from K input data, a regression function in a function space of high dimension N using projections onto a random subspace of lower dimension M. From any linear approximation algorithm…

regression

Near Optimal Sketching of Low-Rank Tensor Regression

2017-09-20 · NeurIPS 2017 12 · Jarvis Haupt, Xingguo Li, David P. Woodruff

We study the least squares regression problem \begin{align*} \min_{\Theta \in \mathcal{S}_{\odot D,R}} \|A\Theta-b\|_2, \end{align*} where $\mathcal{S}_{\odot D,R}$ is the set of $\Theta$ for which $\Theta = \sum_{r=1}^{…

Dimensionality Reductionregression

LSTD with Random Projections

2010-12-01 · NeurIPS 2010 12 · Mohammad Ghavamzadeh, Alessandro Lazaric, Odalric Maillard, Rémi Munos

We consider the problem of reinforcement learning in high-dimensional spaces when the number of features is bigger than the number of samples. In particular, we study the least-squares temporal difference (LSTD) learning…

reinforcement-learningReinforcement LearningReinforcement Learning (RL)

No penalty no tears: Least squares in high-dimensional linear models

2015-06-07 · Xiangyu Wang, David Dunson, Chenlei Leng

Ordinary least squares (OLS) is the default method for fitting linear models, but is not applicable for problems with dimensionality larger than the sample size. For these problems, we advocate the use of a generalized v…

regressionVocal Bursts Intensity Prediction