Explaining the Road Not Taken
It is unclear if existing interpretations of deep neural network models respond effectively to the needs of users. This paper summarizes the common forms of explanations (such as feature attribution, decision rules, or probes) used in over 200 recent papers about natural language processing (NLP), and compares them against user questions collected in the XAI Question Bank. We found that although users are interested in explanations for the road not taken -- namely, why the model chose one result and not a well-defined, seemly similar legitimate counterpart -- most model interpretations cannot answer these questions.
Code (0)
등록된 구현이 없습니다.
Tasks
Explainable Artificial Intelligence (XAI)Similar Papers 제목 키워드 기반
Strategically Linked Decisions in Long-Term Planning and Reinforcement Learning
Long-term planning, as in reinforcement learning (RL), involves finding strategies: actions that collectively work toward a goal rather than individually optimizing their immediate outcomes. As part of a strategy, some a…
Reinforcement Learning (RL)Shapley Values of Reconstruction Errors of PCA for Explaining Anomaly Detection
We present a method to compute the Shapley values of reconstruction errors of principal component analysis (PCA), which is particularly useful in explaining the results of anomaly detection based on PCA. Because features…
Anomaly DetectionRaw reconstructionSelf-Explaining Deviations for Coordination
Fully cooperative, partially observable multi-agent problems are ubiquitous in the real world. In this paper, we focus on a specific subclass of coordination problems in which humans are able to discover self-explaining …
Learning to Act Properly: Predicting and Explaining Affordances from Images
We address the problem of affordance reasoning in diverse scenes that appear in the real world. Affordances relate the agent's actions to their effects when taken on the surrounding objects. In our work, we take the egoc…
Visual ReasoningTowards Transparent Robotic Planning via Contrastive Explanations
Providing explanations of chosen robotic actions can help to increase the transparency of robotic planning and improve users' trust. Social sciences suggest that the best explanations are contrastive, explaining not just…