Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting
When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose of the current paper is to develop a bootstrap method for solving this problem in the context of regression --- which complements our companion paper in the context of classification (Lopes 2019). In contrast to the classification setting, the current paper shows that theoretical guarantees for the proposed bootstrap can be established under much weaker assumptions. In addition, we illustrate the flexibility of the method by showing how it can be adapted to measure algorithmic convergence for variable selection. Lastly, we provide numerical results demonstrating that the method works well in a range of situations.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationregressionVariable SelectionSimilar Papers 제목 키워드 기반
Estimating the Algorithmic Variance of Randomized Ensembles via the Bootstrap
Although the methods of bagging and random forests are some of the most widely used prediction methods, relatively little is known about their algorithmic convergence. In particular, there are not many theoretical guaran…
Estimating a sharp convergence bound for randomized ensembles
When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the pr…
Binary ClassificationDensity EstimationPredictionDon't Explain Noise: Robust Counterfactuals for Randomized Ensembles
Counterfactual explanations describe how to modify a feature vector in order to flip the outcome of a trained classifier. Obtaining robust counterfactual explanations is essential to provide valid algorithmic recourse an…
counterfactualvalidAsymptotically free sketched ridge ensembles: Risks, cross-validation, and tuning
We employ random matrix theory to establish consistency of generalized cross validation (GCV) for estimating prediction risks of sketched ridge regression ensembles, enabling efficient and consistent tuning of regulariza…
PredictionPrediction IntervalsregressionRows vs Columns for Linear Systems of Equations - Randomized Kaczmarz or Coordinate Descent?
This paper is about randomized iterative algorithms for solving a linear system of equations $X \beta = y$ in different settings. Recent interest in the topic was reignited when Strohmer and Vershynin (2009) proved the l…