Sequential Explanations with Mental Model-Based Policies
The act of explaining across two parties is a feedback loop, where one provides information on what needs to be explained and the other provides an explanation relevant to this information. We apply a reinforcement learning framework which emulates this format by providing explanations based on the explainee's current mental model. We conduct novel online human experiments where explanations generated by various explanation methods are selected and presented to participants, using policies which observe participants' mental models, in order to optimize an interpretability proxy. Our results suggest that mental model-based policies (anchored in our proposed state representation) may increase interpretability over multiple sequential explanations, when compared to a random selection baseline. This work provides insight into how to select explanations which increase relevant information for users, and into conducting human-grounded experimentation to understand interpretability.
Code (0)
등록된 구현이 없습니다.
Tasks
modelMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning impartial policies for sequential counterfactual explanations using Deep Reinforcement Learning
In the field of explainable Artificial Intelligence (XAI), sequential counterfactual (SCF) examples are often used to alter the decision of a trained classifier by implementing a sequence of modifications to the input in…
counterfactualDeep Reinforcement LearningExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)+2Explaining Decentralized Multi-Agent Reinforcement Learning Policies
Multi-Agent Reinforcement Learning (MARL) has gained significant interest in recent years, enabling sequential decision-making across multiple agents in various domains. However, most existing explanation methods focus o…
Multi-agent Reinforcement LearningComputational EfficiencyExplainability in AI Policies: A Critical Review of Communications, Reports, Regulations, and Standards in the EU, US, and UK
Public attention towards explainability of artificial intelligence (AI) systems has been rising in recent years to offer methodologies for human oversight. This has translated into the proliferation of research outputs, …
One-shot Policy Elicitation via Semantic Reward Manipulation
Synchronizing expectations and knowledge about the state of the world is an essential capability for effective collaboration. For robots to effectively collaborate with humans and other autonomous agents, it is critical …
Harnessing the Power of Explanations for Incremental Training: A LIME-Based Approach
Explainability of neural network prediction is essential to understand feature importance and gain interpretable insight into neural network performance. However, explanations of neural network outcomes are mostly limite…
Feature ImportanceIncremental LearningKeyword Spotting