Unifying Nonlocal Blocks for Neural Networks
The nonlocal-based blocks are designed for capturing long-range spatial-temporal dependencies in computer vision tasks. Although having shown excellent performance, they still lack the mechanism to encode the rich, structured information among elements in an image or video. In this paper, to theoretically analyze the property of these nonlocal-based blocks, we provide a new perspective to interpret them, where we view them as a set of graph filters generated on a fully-connected graph. Specifically, when choosing the Chebyshev graph filter, a unified formulation can be derived for explaining and analyzing the existing nonlocal-based blocks (e.g., nonlocal block, nonlocal stage, double attention block). Furthermore, by concerning the property of spectral, we propose an efficient and robust spectral nonlocal block, which can be more robust and flexible to catch long-range dependencies when inserted into deep neural networks than the existing nonlocal blocks. Experimental results demonstrate the clear-cut improvements and practical applicabilities of our method on image classification, action recognition, semantic segmentation, and person re-identification tasks.
Code (1)
Tasks
Action Recognitionimage-classificationImage ClassificationPerson Re-IdentificationSemantic SegmentationSimilar Papers 제목 키워드 기반
A Unified Framework to Analyze and Design the Nonlocal Blocks for Neural Networks
The nonlocal-based blocks are designed for capturing long-range spatial-temporal dependencies in computer vision tasks. Although having shown excellent performances, they lack the mechanism to encode the rich, structured…
Action RecognitionFine-Grained Image Classificationimage-classificationImage ClassificationA Spectral Nonlocal Block for Neural Networks
The nonlocal-based blocks are designed for capturing long-range spatial-temporal dependencies in computer vision tasks. Although having shown excellent performances, they lack the mechanism to encode the rich, structured…
Action RecognitionFine-Grained Image ClassificationGeneral Classificationimage-classification+3Spectral Nonlocal Block for Neural Network
The nonlocal network is designed for capturing long-range spatial-temporal dependencies in several computer vision tasks. Although having shown excellent performances, it needs an elaborate preparation for both the numbe…
Video ClassificationSuper-Resolution of Brain MRI Images using Overcomplete Dictionaries and Nonlocal Similarity
Recently, the Magnetic Resonance Imaging (MRI) images have limited and unsatisfactory resolutions due to various constraints such as physical, technological and economic considerations. Super-resolution techniques can ob…
Compressive SensingImage Super-ResolutionSuper-ResolutionAccurate and Lightweight Image Super-Resolution with Model-Guided Deep Unfolding Network
Deep neural networks (DNNs) based methods have achieved great success in single image super-resolution (SISR). However, existing state-of-the-art SISR techniques are designed like black boxes lacking transparency and int…
DenoisingImage Super-ResolutionSuper-Resolution