Nonparametric Feature Selection by Random Forests and Deep Neural Networks
Random forests are a widely used machine learning algorithm, but their computational efficiency is undermined when applied to large-scale datasets with numerous instances and useless features. Herein, we propose a nonparametric feature selection algorithm that incorporates random forests and deep neural networks, and its theoretical properties are also investigated under regularity conditions. Using different synthetic models and a real-world example, we demonstrate the advantage of the proposed algorithm over other alternatives in terms of identifying useful features, avoiding useless ones, and the computation efficiency. Although the algorithm is proposed using standard random forests, it can be widely adapted to other machine learning algorithms, as long as features can be sorted accordingly.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningComputational Efficiencyfeature selectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
High-Dimensional Dynamic Covariance Models with Random Forests
This paper introduces a novel nonparametric method for estimating high-dimensional dynamic covariance matrices with multiple conditioning covariates, leveraging random forests and supported by robust theoretical guarante…
Model SelectionNonparametric IPSS: Fast, flexible feature selection with false discovery control
Feature selection is a critical task in machine learning and statistics. However, existing feature selection methods either (i) rely on parametric methods such as linear or generalized linear models, (ii) lack theoretica…
Feature Importancefeature selectionFACT: High-Dimensional Random Forests Inference
Quantifying the usefulness of individual features in random forests learning can greatly enhance its interpretability. Existing studies have shown that some popularly used feature importance measures for random forests s…
Feature ImportanceVocal Bursts Intensity PredictionAsymptotic confidence bands for centered purely random forests
In a multivariate nonparametric regression setting we construct explicit asymptotic uniform confidence bands for centered purely random forests. Since the most popular example in this class of random forests, namely the …
Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Application
We propose a method for transfer learning in nonparametric regression using a random forest (RF) with distance covariance-based feature weights, assuming the unknown source and target regression functions are sparsely di…
Transfer Learning