Spatial-temporal Concept based Explanation of 3D ConvNets
Recent studies have achieved outstanding success in explaining 2D image recognition ConvNets. On the other hand, due to the computation cost and complexity of video data, the explanation of 3D video recognition ConvNets is relatively less studied. In this paper, we present a 3D ACE (Automatic Concept-based Explanation) framework for interpreting 3D ConvNets. In our approach: (1) videos are represented using high-level supervoxels, which is straightforward for human to understand; and (2) the interpreting framework estimates a score for each voxel, which reflects its importance in the decision procedure. Experiments show that our method can discover spatial-temporal concepts of different importance-levels, and thus can explore the influence of the concepts on a target task, such as action classification, in-depth. The codes are publicly available.
Code (1)
Tasks
Action ClassificationVideo RecognitionSimilar Papers 제목 키워드 기반
Initialization Strategies of Spatio-Temporal Convolutional Neural Networks
We propose a new way of incorporating temporal information present in videos into Spatial Convolutional Neural Networks (ConvNets) trained on images, that avoids training Spatio-Temporal ConvNets from scratch. We describ…
Learning Spatiotemporal Features with 3D Convolutional Networks
We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold…
Action RecognitionAction Recognition In VideosDynamic Facial Expression RecognitionText Understanding from Scratch
This article demontrates that we can apply deep learning to text understanding from character-level inputs all the way up to abstract text concepts, using temporal convolutional networks (ConvNets). We apply ConvNets to …
General ClassificationSentiment AnalysisPooling the Convolutional Layers in Deep ConvNets for Action Recognition
Deep ConvNets have shown its good performance in image classification tasks. However it still remains as a problem in deep video representation for action recognition. The problem comes from two aspects: on one hand, cur…
Action Recognitionimage-classificationImage ClassificationTemporal Action LocalizationLanguage-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation
Referring video object segmentation aims to predict foreground labels for objects referred by natural language expressions in videos. Previous methods either depend on 3D ConvNets or incorporate additional 2D ConvNets as…
DenoisingReferring Video Object SegmentationSemantic SegmentationVideo Object Segmentation+1