Explaining Predictive Uncertainty by Exposing Second-Order Effects
Explainable AI has brought transparency into complex ML blackboxes, enabling, in particular, to identify which features these models use for their predictions. So far, the question of explaining predictive uncertainty, i.e. why a model 'doubts', has been scarcely studied. Our investigation reveals that predictive uncertainty is dominated by second-order effects, involving single features or product interactions between them. We contribute a new method for explaining predictive uncertainty based on these second-order effects. Computationally, our method reduces to a simple covariance computation over a collection of first-order explanations. Our method is generally applicable, allowing for turning common attribution techniques (LRP, Gradient x Input, etc.) into powerful second-order uncertainty explainers, which we call CovLRP, CovGI, etc. The accuracy of the explanations our method produces is demonstrated through systematic quantitative evaluations, and the overall usefulness of our method is demonstrated via two practical showcases.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Explaining Predictive Uncertainty by Looking Back at Model Explanations
Predictive uncertainty estimation of pre-trained language models is an important measure of how likely people can trust their predictions. However, little is known about what makes a model prediction uncertain. Explainin…
Decision MakingNatural Language InferenceParaphrase IdentificationPrediction+1Second-Order Uncertainty Quantification: A Distance-Based Approach
In the past couple of years, various approaches to representing and quantifying different types of predictive uncertainty in machine learning, notably in the setting of classification, have been proposed on the basis of …
Uncertainty QuantificationWord-Level Uncertainty Estimation for Black-Box Text Classifiers using RNNs
Estimating uncertainties of Neural Network predictions paves the way towards more reliable and trustful text classifications. However, common uncertainty estimation approaches remain as black-boxes without explaining whi…
Decision MakingSentiment AnalysisExplaining Predictive Uncertainty with Information Theoretic Shapley Values
Researchers in explainable artificial intelligence have developed numerous methods for helping users understand the predictions of complex supervised learning models. By contrast, explaining the $\textit{uncertainty}$ of…
Active LearningDecision Making Under UncertaintyDepth Aleatoric Uncertainty EstimationExplainable Artificial Intelligence (XAI)+1Set-based v.s. Distribution-based Representations of Epistemic Uncertainty: A Comparative Study
Epistemic uncertainty in neural networks is commonly modeled using two second-order paradigms: distribution-based representations, which rely on posterior parameter distributions, and set-based representations based on c…
Out-of-Distribution Detection