Parameter-Free Average Attention Improves Convolutional Neural Network Performance (Almost) Free of Charge
Visual perception is driven by the focus on relevant aspects in the surrounding world. To transfer this observation to the digital information processing of computers, attention mechanisms have been introduced to highlight salient image regions. Here, we introduce a parameter-free attention mechanism called PfAAM, that is a simple yet effective module. It can be plugged into various convolutional neural network architectures with a little computational overhead and without affecting model size. PfAAM was tested on multiple architectures for classification and segmentic segmentation leading to improved model performance for all tested cases. This demonstrates its wide applicability as a general easy-to-use module for computer vision tasks. The implementation of PfAAM can be found on https://github.com/nkoerb/pfaam.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Parameter-Free Channel Attention for Image Classification and Super-Resolution
The channel attention mechanism is a useful technique widely employed in deep convolutional neural networks to boost the performance for image processing tasks, eg, image classification and image super-resolution. It is …
Classificationimage-classificationImage ClassificationImage Super-Resolution+1ConCA: Concentration-Aware Channel Attention for Fine-Grained Visual Recognition
Lightweight channel attention mechanisms are widely used in image classification, yet their effectiveness in fine-grained visual recognition (FGVR) remains limited. Most modules summarize each channel by global average p…
Fine-Grained Visual RecognitionImage ClassificationJoint Spatial and Layer Attention for Convolutional Networks
In this paper, we propose a novel approach that learns to sequentially attend to different Convolutional Neural Networks (CNN) layers (i.e., ``what'' feature abstraction to attend to) and different spatial locations of t…
Camera LocalizationClassificationGeneral ClassificationScene ClassificationParameter-Free Spatial Attention Network for Person Re-Identification
Global average pooling (GAP) allows to localize discriminative information for recognition [40]. While GAP helps the convolution neural network to attend to the most discriminative features of an object, it may suffer if…
Person Re-IdentificationDo Value Vectors in Deep Layers Need Context from the Residual Stream?
The success of the transformer architecture as the backbone of modern LLMs is in large part due to its use of attention layers. An attention layer follows the standard neural network paradigm: it takes the residual strea…