Explaining Explaining
Explanation is key to people having confidence in high-stakes AI systems. However, machine-learning-based systems -- which account for almost all current AI -- can't explain because they are usually black boxes. The explainable AI (XAI) movement hedges this problem by redefining "explanation". The human-centered explainable AI (HCXAI) movement identifies the explanation-oriented needs of users but can't fulfill them because of its commitment to machine learning. In order to achieve the kinds of explanations needed by real people operating in critical domains, we must rethink how to approach AI. We describe a hybrid approach to developing cognitive agents that uses a knowledge-based infrastructure supplemented by data obtained through machine learning when applicable. These agents will serve as assistants to humans who will bear ultimate responsibility for the decisions and actions of the human-robot team. We illustrate the explanatory potential of such agents using the under-the-hood panels of a demonstration system in which a team of simulated robots collaborate on a search task assigned by a human.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Explaining Predictive Uncertainty by Looking Back at Model Explanations
Predictive uncertainty estimation of pre-trained language models is an important measure of how likely people can trust their predictions. However, little is known about what makes a model prediction uncertain. Explainin…
Decision MakingNatural Language InferenceParaphrase IdentificationPrediction+1Self-Explaining Structures Improve NLP Models
Existing approaches to explaining deep learning models in NLP usually suffer from two major drawbacks: (1) the main model and the explaining model are decoupled: an additional probing or surrogate model is used to interp…
Natural Language InferenceParaphrase IdentificationSentiment AnalysisThe Role of Interactive Visualization in Explaining (Large) NLP Models: from Data to Inference
With a constant increase of learned parameters, modern neural language models become increasingly more powerful. Yet, explaining these complex model's behavior remains a widely unsolved problem. In this paper, we discuss…
Case Study: Explaining Diabetic Retinopathy Detection Deep CNNs via Integrated Gradients
In this report, we applied integrated gradients to explaining a neural network for diabetic retinopathy detection. The integrated gradient is an attribution method which measures the contributions of input to the quantit…
Diabetic Retinopathy DetectionInterpretable Text Embeddings and Text Similarity Explanation: A Primer
Text embeddings and text embedding models are a backbone of many AI and NLP systems, particularly those involving search. However, interpretability challenges persist, especially in explaining obtained similarity scores,…
Similarity Explanationtext similarity