Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models
Recent work in XAI for eye tracking data has evaluated the suitability of feature attribution methods to explain the output of deep neural sequence models for the task of oculomotric biometric identification. These methods provide saliency maps to highlight important input features of a specific eye gaze sequence. However, to date, its localization analysis has been lacking a quantitative approach across entire datasets. In this work, we employ established gaze event detection algorithms for fixations and saccades and quantitatively evaluate the impact of these events by determining their concept influence. Input features that belong to saccades are shown to be substantially more important than features that belong to fixations. By dissecting saccade events into sub-events, we are able to show that gaze samples that are close to the saccadic peak velocity are most influential. We further investigate the effect of event properties like saccadic amplitude or fixational dispersion on the resulting concept influence.
Code (1)
Tasks
Event DetectionExplainable Artificial Intelligence (XAI)Similar Papers 제목 키워드 기반
Domain Expansion: A Latent Space Construction Framework for Multi-Task Learning
Training a single network with multiple objectives often leads to conflicting gradients that degrade shared representations, forcing them into a compromised state that is suboptimal for any single task--a problem we term…
Multi-Task LearningGaze EstimationRotated MNISTLeveraging multimodal explanatory annotations for video interpretation with Modality Specific Dataset
We examine the impact of concept-informed supervision on multimodal video interpretation models using MOByGaze, a dataset containing human-annotated explanatory concepts. We introduce Concept Modality Specific Datasets (…
Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis
Models based on human-understandable concepts have received extensive attention to improve model interpretability for trustworthy artificial intelligence in the field of medical image analysis. These methods can provide …
DiagnosticExplainable ModelsMedical Image AnalysisNUM2EVENT: Interpretable Event Reasoning from Numerical time-series
Large language models (LLMs) have recently demonstrated impressive multimodal reasoning capabilities, yet their understanding of purely numerical time-series signals remains limited. Existing approaches mainly focus on f…
Multimodal ReasoningESPRIT: Explaining Solutions to Physical Reasoning Tasks
Neural networks lack the ability to reason about qualitative physics and so cannot generalize to scenarios and tasks unseen during training. We propose ESPRIT, a framework for commonsense reasoning about qualitative phys…