Marginal and Conditional Importance Measures from Machine Learning Models and Their Relationship with Conditional Average Treatment Effect
Interpreting black-box machine learning models is challenging due to their strong dependence on data and inherently non-parametric nature. This paper reintroduces the concept of importance through "Marginal Variable Importance Metric" (MVIM), a model-agnostic measure of predictor importance based on the true conditional expectation function. MVIM evaluates predictors' influence on continuous or discrete outcomes. A permutation-based estimation approach, inspired by \citet{breiman2001random} and \citet{fisher2019all}, is proposed to estimate MVIM. MVIM estimator is biased when predictors are highly correlated, as black-box models struggle to extrapolate in low-probability regions. To address this, we investigated the bias-variance decomposition of MVIM to understand the source and pattern of the bias under high correlation. A Conditional Variable Importance Metric (CVIM), adapted from \citet{strobl2008conditional}, is introduced to reduce this bias. Both MVIM and CVIM exhibit a quadratic relationship with the conditional average treatment effect (CATE).
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Conditional Feature Importance for Mixed Data
Despite the popularity of feature importance (FI) measures in interpretable machine learning, the statistical adequacy of these methods is rarely discussed. From a statistical perspective, a major distinction is between …
Feature ImportanceInterpretable Machine Learningxplainfi: Feature Importance and Statistical Inference for Machine Learning in R
We introduce xplainfi, an R package built on top of the mlr3 ecosystem for global, loss-based feature importance methods for machine learning models. Various feature importance methods exist in R, but significant gaps re…
Feature ImportanceA Generalized Variable Importance Metric and Estimator for Black Box Machine Learning Models
In this paper we define a population parameter, ``Generalized Variable Importance Metric (GVIM)'', to measure importance of predictors for black box machine learning methods, where the importance is not represented by mo…
Systemic Risk: Conditional Distortion Risk Measures
In this paper, we introduce the rich classes of conditional distortion (CoD) risk measures and distortion risk contribution ($\Delta$CoD) measures as measures of systemic risk and analyze their properties and representat…
AttributeTripletVisualizing the Feature Importance for Black Box Models
In recent years, a large amount of model-agnostic methods to improve the transparency, trustability and interpretability of machine learning models have been developed. We introduce local feature importance as a local ve…
Feature Importance