Discriminating Spatial and Temporal Relevance in Deep Taylor Decompositions for Explainable Activity Recognition
Current techniques for explainable AI have been applied with some success to image processing. The recent rise of research in video processing has called for similar work n deconstructing and explaining spatio-temporal models. While many techniques are designed for 2D convolutional models, others are inherently applicable to any input domain. One such body of work, deep Taylor decomposition, propagates relevance from the model output distributively onto its input and thus is not restricted to image processing models. However, by exploiting a simple technique that removes motion information, we show that it is not the case that this technique is effective as-is for representing relevance in non-image tasks. We instead propose a discriminative method that produces a na\"ive representation of both the spatial and temporal relevance of a frame as two separate objects. This new discriminative relevance model exposes relevance in the frame attributed to motion, that was previously ambiguous in the original explanation. We observe the effectiveness of this technique on a range of samples from the UCF-101 action recognition dataset, two of which are demonstrated in this paper.
Code (4)
Tasks
Action RecognitionActivity RecognitionSimilar Papers 제목 키워드 기반
Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
Convolutional Neural Networks (CNN) have become state-of-the-art in the field of image classification. However, not everything is understood about their inner representations. This paper tackles the interpretability and …
AttributeClassificationGeneral Classificationimage-classification+2Spatial-temporal attributes in verbal semantics: A corpus-based lexical semantic study of discriminating Mandarin near synonyms of “tui1” and “la1”
Compressed Online Dictionary Learning for Fast fMRI Decomposition
We present a method for fast resting-state fMRI spatial decomposi-tions of very large datasets, based on the reduction of the temporal dimension before applying dictionary learning on concatenated individual records from…
Dictionary LearningSummable Reparameterizations of Wasserstein Critics in the One-Dimensional Setting
Generative adversarial networks (GANs) are an exciting alternative to algorithms for solving density estimation problems---using data to assess how likely samples are to be drawn from the same distribution. Instead of ex…
Density EstimationTensor Decompositions in Deep Learning
The paper surveys the topic of tensor decompositions in modern machine learning applications. It focuses on three active research topics of significant relevance for the community. After a brief review of consolidated wo…
BIG-bench Machine LearningDeep Learning