Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines
A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using randomized decision trees of depth 1, which provide a very simple architecture. Weights of GBMs as well as features are computed in each iteration of boosting by using the Lasso method and then updated by means of a specific smoothing procedure. In contrast to the neural additive model, the method provides weights of features in the explicit form, and it is simply trained. A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation.
Code (1)
Tasks
Additive modelsBIG-bench Machine LearningInterpretable Machine LearningSimilar Papers 제목 키워드 기반
Instance-based Explanations for Gradient Boosting Machine Predictions with AXIL Weights
We show that regression predictions from linear and tree-based models can be represented as linear combinations of target instances in the training data. This also holds for models constructed as ensembles of trees, incl…
Feature ImportanceregressionA Generalized Stacking for Implementing Ensembles of Gradient Boosting Machines
The gradient boosting machine is one of the powerful tools for solving regression problems. In order to cope with its shortcomings, an approach for constructing ensembles of gradient boosting models is proposed. The main…
regressionInherently Interpretable Tree Ensemble Learning
Tree ensemble models like random forests and gradient boosting machines are widely used in machine learning due to their excellent predictive performance. However, a high-performance ensemble consisting of a large number…
Additive modelsEnsemble LearningCalibrated Boosting-Forest
Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel techniq…
Deep LearningGeneral ClassificationregressionProbabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression
Gradient Boosting Machines (GBM) are hugely popular for solving tabular data problems. However, practitioners are not only interested in point predictions, but also in probabilistic predictions in order to quantify the u…
regressionTime Series Analysis