Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition
Explainable machine learning and artificial intelligence models have been used to justify a model's decision-making process. This added transparency aims to help improve user performance and understanding of the underlying model. However, in practice, explainable systems face many open questions and challenges. Specifically, designers might reduce the complexity of deep learning models in order to provide interpretability. The explanations generated by these simplified models, however, might not accurately justify and be truthful to the model. This can further add confusion to the users as they might not find the explanations meaningful with respect to the model predictions. Understanding how these explanations affect user behavior is an ongoing challenge. In this paper, we explore how explanation veracity affects user performance and agreement in intelligent systems. Through a controlled user study with an explainable activity recognition system, we compare variations in explanation veracity for a video review and querying task. The results suggest that low veracity explanations significantly decrease user performance and agreement compared to both accurate explanations and a system without explanations. These findings demonstrate the importance of accurate and understandable explanations and caution that poor explanations can sometimes be worse than no explanations with respect to their effect on user performance and reliance on an AI system.
Code (0)
등록된 구현이 없습니다.
Tasks
Activity RecognitionDecision MakingSimilar Papers 제목 키워드 기반
Explainable Automated Fact-Checking for Public Health Claims
Fact-checking is the task of verifying the veracity of claims by assessing their assertions against credible evidence. The vast majority of fact-checking studies focus exclusively on political claims. Very little researc…
Explanation GenerationFact CheckingA Graph-Enhanced Defense Framework for Explainable Fake News Detection with LLM
Explainable fake news detection aims to assess the veracity of news claims while providing human-friendly explanations. Existing methods incorporating investigative journalism are often inefficient and struggle with brea…
Explanation GenerationFake News DetectionConsistent Multi-Granular Rationale Extraction for Explainable Multi-hop Fact Verification
The success of deep learning models on multi-hop fact verification has prompted researchers to understand the behavior behind their veracity. One possible way is erasure search: obtaining the rationale by entirely removi…
DiagnosticFact VerificationSentenceTell Me Why: Explainable Public Health Fact-Checking with Large Language Models
This paper presents a comprehensive analysis of explainable fact-checking through a series of experiments, focusing on the ability of large language models to verify public health claims and provide explanations or justi…
Explanation GenerationFact Checkingparameter-efficient fine-tuningAMREx: AMR for Explainable Fact Verification
With the advent of social media networks and the vast amount of information circulating through them, automatic fact verification is an essential component to prevent the spread of misinformation. It is even more useful …
Abstract Meaning RepresentationClaim VerificationFact VerificationMisinformation