Cost-complexity pruning of random forests
Random forests perform bootstrap-aggregation by sampling the training samples with replacement. This enables the evaluation of out-of-bag error which serves as a internal cross-validation mechanism. Our motivation lies in using the unsampled training samples to improve each decision tree in the ensemble. We study the effect of using the out-of-bag samples to improve the generalization error first of the decision trees and second the random forest by post-pruning. A preliminary empirical study on four UCI repository datasets show consistent decrease in the size of the forests without considerable loss in accuracy.
Code (1)
Similar Papers 제목 키워드 기반
Dynamic Connected Neural Decision Classifier and Regressor with Dynamic Softing Pruning
To deal with various datasets over different complexity, this paper presents an self-adaptive learning model that combines the proposed Dynamic Connected Neural Decision Networks (DNDN) and a new pruning method--Dynamic …
ClassificationGeneral ClassificationRepresentation LearningAMF: Aggregated Mondrian Forests for Online Learning
Random Forests (RF) is one of the algorithms of choice in many supervised learning applications, be it classification or regression. The appeal of such tree-ensemble methods comes from a combination of several characteri…
General ClassificationMulti-class ClassificationregressionImproving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement
Random Forests (RF) are among the state-of-the-art in many machine learning applications. With the ongoing integration of ML models into everyday life, the deployment and continuous application of models becomes more and…
Ensemble PruningPruning Random Forests for Prediction on a Budget
We propose to prune a random forest (RF) for resource-constrained prediction. We first construct a RF and then prune it to optimize expected feature cost & accuracy. We pose pruning RFs as a novel 0-1 integer program wit…
Combinatorial OptimizationPredictionPartitioning Structure Learning for Segmented Linear Regression Trees
This paper proposes a partitioning structure learning method for segmented linear regression trees (SLRT), which assigns linear predictors over the terminal nodes. The recursive partitioning process is driven by an adapt…
regression