Thresholding Procedures for High Dimensional Variable Selection and Statistical Estimation
Given $n$ noisy samples with $p$ dimensions, where $n \ll p$, we show that the multi-stage thresholding procedures can accurately estimate a sparse vector $\beta \in \R^p$ in a linear model, under the restricted eigenvalue conditions (Bickel-Ritov-Tsybakov 09). Thus our conditions for model selection consistency are considerably weaker than what has been achieved in previous works. More importantly, this method allows very significant values of $s$, which is the number of non-zero elements in the true parameter $\beta$. For example, it works for cases where the ordinary Lasso would have failed. Finally, we show that if $X$ obeys a uniform uncertainty principle and if the true parameter is sufficiently sparse, the Gauss-Dantzig selector (Cand\{e}s-Tao 07) achieves the $\ell_2$ loss within a logarithmic factor of the ideal mean square error one would achieve with an oracle which would supply perfect information about which coordinates are non-zero and which are above the noise level, while selecting a sufficiently sparse model.
Code (0)
등록된 구현이 없습니다.
Tasks
Model SelectionVariable SelectionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Controlling false discoveries in high-dimensional situations: Boosting with stability selection
Modern biotechnologies often result in high-dimensional data sets with much more variables than observations (n $\ll$ p). These data sets pose new challenges to statistical analysis: Variable selection becomes one of the…
Variable SelectionVocal Bursts Intensity PredictionOn feature selection in double-imbalanced data settings: a Random Forest approach
Feature selection is a critical step in high-dimensional classification tasks, particularly under challenging conditions of double imbalance, namely settings characterized by both class imbalance in the response variable…
feature selectionVariable SelectionPath Thresholding: Asymptotically Tuning-Free High-Dimensional Sparse Regression
In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression. We propose a simple and computationally efficient method, called path thresholding (PaTh), that tra…
Model SelectionregressionVocal Bursts Intensity PredictionEffect of hyperparameters on variable selection in random forests
Random forests (RFs) are well suited for prediction modeling and variable selection in high-dimensional omics studies. The effect of hyperparameters of the RF algorithm on prediction performance and variable importance e…
SensitivityVariable SelectionMDFS - MultiDimensional Feature Selection
Identification of informative variables in an information system is often performed using simple one-dimensional filtering procedures that discard information about interactions between variables. Such approach may resul…
feature selection