Approximate False Positive Rate Control in Selection Frequency for Random Forest
Random Forest has become one of the most popular tools for feature selection. Its ability to deal with high-dimensional data makes this algorithm especially useful for studies in neuroimaging and bioinformatics. Despite its popularity and wide use, feature selection in Random Forest still lacks a crucial ingredient: false positive rate control. To date there is no efficient, principled and computationally light-weight solution to this shortcoming. As a result, researchers using Random Forest for feature selection have to resort to using heuristically set thresholds on feature rankings. This article builds an approximate probabilistic model for the feature selection process in random forest training, which allows us to compute an estimated false positive rate for a given threshold on selection frequency. Hence, it presents a principled way to determine thresholds for the selection of relevant features without any additional computational load. Experimental analysis with synthetic data demonstrates that the proposed approach can limit false positive rates on the order of the desired values and keep false negative rates low. Results show that this holds even in the presence of a complex correlation structure between features. Its good statistical properties and light-weight computational needs make this approach widely applicable to feature selection for a wide-range of applications.
Code (1)
Tasks
feature selectionSimilar Papers 제목 키워드 기반
Information criteria for structured parameter selection in high dimensional tree and graph models
Parameter selection in high-dimensional models is typically finetuned in a way that keeps the (relative) number of false positives under control. This is because otherwise the few true positives may be dominated by the m…
Integrated path stability selection
Stability selection is a popular method for improving feature selection algorithms. One of its key attributes is that it provides theoretical upper bounds on the expected number of false positives, E(FP), enabling contro…
feature selectionNonparametric IPSS: Fast, flexible feature selection with false discovery control
Feature selection is a critical task in machine learning and statistics. However, existing feature selection methods either (i) rely on parametric methods such as linear or generalized linear models, (ii) lack theoretica…
Feature Importancefeature selectionStability Selection for Structured Variable Selection
In variable or graph selection problems, finding a right-sized model or controlling the number of false positives is notoriously difficult. Recently, a meta-algorithm called Stability Selection was proposed that can prov…
Variable SelectionKernel Stein Tests for Multiple Model Comparison
We address the problem of non-parametric multiple model comparison: given $l$ candidate models, decide whether each candidate is as good as the best one(s) or worse than it. We propose two statistical tests, each control…
model