People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Explainable AI
It is often argued that effective human-centered explainable artificial intelligence (XAI) should resemble human reasoning. However, empirical investigations of how concepts from cognitive science can aid the design of XAI are lacking. Based on insights from cognitive science, we propose a framework of explanatory modes to analyze how people frame explanations, whether mechanistic, teleological, or counterfactual. Using the complex safety-critical domain of autonomous driving, we conduct an experiment consisting of two studies on (i) how people explain the behavior of a vehicle in 14 unique scenarios (N1=54) and (ii) how they perceive these explanations (N2=382), curating the novel Human Explanations for Autonomous Driving Decisions (HEADD) dataset. Our main finding is that participants deem teleological explanations significantly better quality than counterfactual ones, with perceived teleology being the best predictor of perceived quality. Based on our results, we argue that explanatory modes are an important axis of analysis when designing and evaluating XAI and highlight the need for a principled and empirically grounded understanding of the cognitive mechanisms of explanation. The HEADD dataset and our code are available at: https://datashare.ed.ac.uk/handle/10283/8930.
Code (1)
Tasks
AttributeAutonomous DrivingAutonomous VehiclescounterfactualDecision MakingExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)Similar Papers 제목 키워드 기반
Attribute Annotation and Bias Evaluation in Visual Datasets for Autonomous Driving
This paper addresses the often overlooked issue of fairness in the autonomous driving domain, particularly in vision-based perception and prediction systems, which play a pivotal role in the overall functioning of Autono…
AttributeAutonomous DrivingAutonomous VehiclesFairness+1Detecting 32 Pedestrian Attributes for Autonomous Vehicles
Pedestrians are arguably one of the most safety-critical road users to consider for autonomous vehicles in urban areas. In this paper, we address the problem of jointly detecting pedestrians and recognizing 32 pedestrian…
AttributeAutonomous DrivingAutonomous VehiclesMulti-Task LearningHuman intuition as a defense against attribute inference
Attribute inference - the process of analyzing publicly available data in order to uncover hidden information - has become a major threat to privacy, given the recent technological leap in machine learning. One way to ta…
AttributeExploring reinforcement learning for incident response in autonomous military vehicles
Unmanned vehicles able to conduct advanced operations without human intervention are being developed at a fast pace for many purposes. Not surprisingly, they are also expected to significantly change how military operati…
reinforcement-learningReinforcement LearningTo Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles
Explainable AI, in the context of autonomous systems, like self-driving cars, has drawn broad interests from researchers. Recent studies have found that providing explanations for autonomous vehicles' actions has many be…
Autonomous VehiclesSelf-Driving Cars