Dissonance Between Human and Machine Understanding
Complex machine learning models are deployed in several critical domains including healthcare and autonomous vehicles nowadays, albeit as functional black boxes. Consequently, there has been a recent surge in interpreting decisions of such complex models in order to explain their actions to humans. Models that correspond to human interpretation of a task are more desirable in certain contexts and can help attribute liability, build trust, expose biases and in turn build better models. It is, therefore, crucial to understand how and which models conform to human understanding of tasks. In this paper, we present a large-scale crowdsourcing study that reveals and quantifies the dissonance between human and machine understanding, through the lens of an image classification task. In particular, we seek to answer the following questions: Which (well-performing) complex ML models are closer to humans in their use of features to make accurate predictions? How does task difficulty affect the feature selection capability of machines in comparison to humans? Are humans consistently better at selecting features that make image recognition more accurate? Our findings have important implications on human-machine collaboration, considering that a long term goal in the field of artificial intelligence is to make machines capable of learning and reasoning like humans.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeAutonomous Vehiclesfeature selectionimage-classificationImage ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Losses, Dissonances, and Distortions
In this paper I present a study in using the losses and gradients obtained during the training of a simple function approximator as a mechanism for creating musical dissonance and visual distortion in a solo piano perfor…
Characterizing Social Imaginaries and Self-Disclosures of Dissonance in Online Conspiracy Discussion Communities
Online discussion platforms offer a forum to strengthen and propagate belief in misinformed conspiracy theories. Yet, they also offer avenues for conspiracy theorists to express their doubts and experiences of cognitive …
2kTopic-independent Detection of Dissonance in Short Stance Text
We address dissonance detection, the task of detecting conflicting stance between two input statements. Computational models for stance detection have typically been trained for a given target topic (e.g. gun control). I…
Stance DetectionEmpirical Evaluation of Topic Zero- and Few-Shot Learning for Stance Dissonance Detection
We address stance dissonance detection, the task of detecting conflicting stance between two input statements. Computational models for traditional stance detection have typically been trained to indicate pro/cons for a …
Few-Shot LearningStance DetectionVisualizing Uncertainty: Spatial Maps of Missing and Conflicting Evidence in Deep Learning
Understanding when and why deep neural networks are uncertain is crucial for deploying reliable machine learning systems in safety-critical domains. While existing uncertainty quantification methods provide scalar measur…