Hypothesis Testing and Machine Learning: Interpreting Variable Effects in Deep Artificial Neural Networks using Cohen's f2
Deep artificial neural networks show high predictive performance in many fields, but they do not afford statistical inferences and their black-box operations are too complicated for humans to comprehend. Because positing that a relationship exists is often more important than prediction in scientific experiments and research models, machine learning is far less frequently used than inferential statistics. Additionally, statistics calls for improving the test of theory by showing the magnitude of the phenomena being studied. This article extends current XAI methods and develops a model agnostic hypothesis testing framework for machine learning. First, Fisher's variable permutation algorithm is tweaked to compute an effect size measure equivalent to Cohen's f2 for OLS regression models. Second, the Mann-Kendall test of monotonicity and the Theil-Sen estimator is applied to Apley's accumulated local effect plots to specify a variable's direction of influence and statistical significance. The usefulness of this approach is demonstrated on an artificial data set and a social survey with a Python sandbox implementation.
Code (0)
등록된 구현이 없습니다.
Tasks
Explainable Artificial Intelligence (XAI)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Inference on effect size after multiple hypothesis testing
Significant treatment effects are often emphasized when interpreting and summarizing empirical findings in studies that estimate multiple, possibly many, treatment effects. Under this kind of selective reporting, convent…
validExploration, inference and prediction in neuroscience and biomedicine
The last decades saw dramatic progress in brain research. These advances were often buttressed by probing single variables to make circumscribed discoveries, typically through null hypothesis significance testing. New wa…
PredictionFunction Trees: Transparent Machine Learning
The output of a machine learning algorithm can usually be represented by one or more multivariate functions of its input variables. Knowing the global properties of such functions can help in understanding the system tha…
Causal Structural Hypothesis Testing and Data Generation Models
A vast amount of expert and domain knowledge is captured by causal structural priors, yet there has been little research on testing such priors for generalization and data synthesis purposes. We propose a novel model arc…
Out-of-Distribution GeneralizationTesting for homogeneous treatment effects in linear and nonparametric instrumental variable models
The hypothesis of homogeneous treatment effects is central to the instrumental variables literature. This assumption signifies that treatment effects are constant across all subjects. It allows to interpret instrumental …