Unbiased variable importance for random forests
The default variable-importance measure in random Forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationally demanding and suffers from other shortcomings. We propose a simple solution to the misleading/untrustworthy Gini importance which can be viewed as an overfitting problem: we compute the loss reduction on the out-of-bag instead of the in-bag training samples.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Unbiased split variable selection for random survival forests using maximally selected rank statistics
The most popular approach for analyzing survival data is the Cox regression model. The Cox model may, however, be misspecified, and its proportionality assumption may not always be fulfilled. An alternative approach for …
Selection biasSurvival PredictionVariable SelectionUnbiased Measurement of Feature Importance in Tree-Based Methods
We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of f…
Feature ImportanceFrom unbiased MDI Feature Importance to Explainable AI for Trees
We attempt to give a unifying view of the various recent attempts to (i) improve the interpretability of tree-based models and (ii) debias the the default variable-importance measure in random Forests, Gini importance. I…
Feature ImportanceUnderstanding variable importances in forests of randomized trees
Despite growing interest and practical use in various scientific areas, variable importances derived from tree-based ensemble methods are not well understood from a theoretical point of view. In this work we characterize…
Trees, forests, and impurity-based variable importance
Tree ensemble methods such as random forests [Breiman, 2001] are very popular to handle high-dimensional tabular data sets, notably because of their good predictive accuracy. However, when machine learning is used for de…
Decision Making