The Explanation Game: Towards Prediction Explainability through Sparse Communication
Explainability is a topic of growing importance in NLP. In this work, we provide a unified perspective of explainability as a communication problem between an explainer and a layperson about a classifier's decision. We use this framework to compare several prior approaches for extracting explanations, including gradient methods, representation erasure, and attention mechanisms, in terms of their communication success. In addition, we reinterpret these methods at the light of classical feature selection, and we use this as inspiration to propose new embedded methods for explainability, through the use of selective, sparse attention. Experiments in text classification, natural language entailment, and machine translation, using different configurations of explainers and laypeople (including both machines and humans), reveal an advantage of attention-based explainers over gradient and erasure methods. Furthermore, human evaluation experiments show promising results with post-hoc explainers trained to optimize communication success and faithfulness.
Code (1)
Tasks
feature selectionMachine TranslationPredictiontext-classificationText ClassificationTranslationSimilar Papers 제목 키워드 기반
The Weighting Game: Evaluating Quality of Explainability Methods
The objective of this paper is to assess the quality of explanation heatmaps for image classification tasks. To assess the quality of explainability methods, we approach the task through the lens of accuracy and stabilit…
image-classificationImage ClassificationGISExplainer: On Explainability of Graph Neural Networks via Game-theoretic Interaction Subgraphs
Explainability is crucial for the application of black-box Graph Neural Networks (GNNs) in critical fields such as healthcare, finance, cybersecurity, and more. Various feature attribution methods, especially the perturb…
Computational EfficiencyNode ClassificationSyntaxShap: Syntax-aware Explainability Method for Text Generation
To harness the power of large language models in safety-critical domains, we need to ensure the explainability of their predictions. However, despite the significant attention to model interpretability, there remains an …
Text GenerationExplanation Beyond Intuition: A Testable Criterion for Inherent Explainability
Inherent explainability is the gold standard in Explainable Artificial Intelligence (XAI). However, there is not a consistent definition or test to demonstrate inherent explainability. Work to date either characterises e…
Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models
Although a recent shift has been made in the field of predictive process monitoring to use models from the explainable artificial intelligence field, the evaluation still occurs mainly through performance-based metrics, …
Explainable artificial intelligencePredictive Process Monitoring