Top-down Neural Attention by Excitation Backprop
We aim to model the top-down attention of a Convolutional Neural Network (CNN) classifier for generating task-specific attention maps. Inspired by a top-down human visual attention model, we propose a new backpropagation scheme, called Excitation Backprop, to pass along top-down signals downwards in the network hierarchy via a probabilistic Winner-Take-All process. Furthermore, we introduce the concept of contrastive attention to make the top-down attention maps more discriminative. In experiments, we demonstrate the accuracy and generalizability of our method in weakly supervised localization tasks on the MS COCO, PASCAL VOC07 and ImageNet datasets. The usefulness of our method is further validated in the text-to-region association task. On the Flickr30k Entities dataset, we achieve promising performance in phrase localization by leveraging the top-down attention of a CNN model that has been trained on weakly labeled web images.
Code (3)
Similar Papers 제목 키워드 기반
Weight Excitation: Built-in Attention Mechanisms in Convolutional Neural Networks
We propose novel approaches for simultaneously identifying important weights of a convolutional neural network (ConvNet) and providing more attention to the important weights during training. More formally, we identify t…
3D Action Recognition3D Classification3D Object Classification3D Semantic Segmentation+9Neuromodulated Goal-Driven Perception in Uncertain Domains
In uncertain domains, the goals are often unknown and need to be predicted by the organism or system. In this paper, contrastive excitation backprop (c-EB) was used in a goal-driven perception task with pairs of noisy MN…
validUnderstanding CNNs from excitations
Saliency maps have proven to be a highly efficacious approach for explicating the decisions of Convolutional Neural Networks. However, extant methodologies predominantly rely on gradients, which constrain their ability t…
Binary ClassificationExcitation Backprop for RNNs
Deep models are state-of-the-art for many vision tasks including video action recognition and video captioning. Models are trained to caption or classify activity in videos, but little is known about the evidence used to…
Action RecognitionTemporal Action LocalizationVideo CaptioningEnhancing the Performance of Transformer-based Spiking Neural Networks by SNN-optimized Downsampling with Precise Gradient Backpropagation
Deep spiking neural networks (SNNs) have drawn much attention in recent years because of their low power consumption, biological rationality and event-driven property. However, state-of-the-art deep SNNs (including Spikf…