Analyze Additive and Interaction Effects via Collaborative Trees
We present Collaborative Trees, a novel tree model designed for regression prediction, along with its bagging version, which aims to analyze complex statistical associations between features and uncover potential patterns inherent in the data. We decompose the mean decrease in impurity from the proposed tree model to analyze the additive and interaction effects of features on the response variable. Additionally, we introduce network diagrams to visually depict how each feature contributes additively to the response and how pairs of features contribute interaction effects. Through a detailed demonstration using an embryo growth dataset, we illustrate how the new statistical tools aid data analysis, both visually and numerically. Moreover, we delve into critical aspects of tree modeling, such as prediction performance, inference stability, and bias in feature importance measures, leveraging real datasets and simulation experiments for comprehensive discussions. On the theory side, we show that Collaborative Trees, built upon a ``sum of trees'' approach with our own innovative tree model regularization, exhibit characteristics akin to matching pursuit, under the assumption of high-dimensional independent binary input features (or one-hot feature groups). This newfound link sheds light on the superior capability of our tree model in estimating additive effects of features, a crucial factor for accurate interaction effect estimation.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature ImportanceSimilar Papers 제목 키워드 기반
Variable Selection with Random Survival Forest and Bayesian Additive Regression Tree for Survival Data
In this paper we utilize a survival analysis methodology incorporating Bayesian additive regression trees to account for nonlinear and additive covariate effects. We compare the performance of Bayesian additive regressio…
regressionSurvival AnalysisVariable SelectionHierarchical Embedded Bayesian Additive Regression Trees
We propose a simple yet powerful extension of Bayesian Additive Regression Trees which we name Hierarchical Embedded BART (HE-BART). The model allows for random effects to be included at the terminal node level of a set …
regressionFast Estimation of Partial Dependence Functions using Trees
Many existing interpretation methods are based on Partial Dependence (PD) functions that, for a pre-trained machine learning model, capture how a subset of the features affects the predictions by averaging over the remai…
Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles
While shallow decision trees may be interpretable, larger ensemble models like gradient-boosted trees, which often set the state of the art in machine learning problems involving tabular data, still remain black box mode…
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)Explainable Recommendation Systems by Generalized Additive Models with Manifest and Latent Interactions
In recent years, the field of recommendation systems has attracted increasing attention to developing predictive models that provide explanations of why an item is recommended to a user. The explanations can be either ob…
Additive modelsCollaborative FilteringExplainable RecommendationRecommendation Systems