A causal framework for explaining the predictions of black-box sequence-to-sequence models
We interpret the predictions of any black-box structured input-structured output model around a specific input-output pair. Our method returns an "explanation" consisting of groups of input-output tokens that are causally related. These dependencies are inferred by querying the black-box model with perturbed inputs, generating a graph over tokens from the responses, and solving a partitioning problem to select the most relevant components. We focus the general approach on sequence-to-sequence problems, adopting a variational autoencoder to yield meaningful input perturbations. We test our method across several NLP sequence generation tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution
With the increasing adoption of predictive models trained using machine learning across a wide range of high-stakes applications, e.g., health care, security, criminal justice, finance, and education, there is a growing …
severity predictionCMA-R:Causal Mediation Analysis for Explaining Rumour Detection
We apply causal mediation analysis to explain the decision-making process of neural models for rumour detection on Twitter. Interventions at the input and network level reveal the causal impacts of tweets and words in th…
Decision MakingRumour DetectionExplaining the Behavior of Black-Box Prediction Algorithms with Causal Learning
Causal approaches to post-hoc explainability for black-box prediction models (e.g., deep neural networks trained on image pixel data) have become increasingly popular. However, existing approaches have two important shor…
counterfactualTimeSAE: Causal Sparse Decoding for Faithful Explanations of Black-Box Time Series Models
As black box models and pretrained models gain traction in time series applications, understanding and explaining their predictions becomes increasingly vital, especially in high-stakes domains where interpretability and…
CIExplainer++: Generating Causal and Interpretable Explanations for Graph Neural Networks
Explainable Artificial Intelligence aims to make black-box models more trustworthy by presenting, in a human-understandable manner, the elements that lead to the model's output. This involves both (i) identifying compone…
Causal Inference