Symmetry-Aware Transformer-based Mirror Detection
Mirror detection aims to identify the mirror regions in the given input image. Existing works mainly focus on integrating the semantic features and structural features to mine specific relations between mirror and non-mirror regions, or introducing mirror properties like depth or chirality to help analyze the existence of mirrors. In this work, we observe that a real object typically forms a loose symmetry relationship with its corresponding reflection in the mirror, which is beneficial in distinguishing mirrors from real objects. Based on this observation, we propose a dual-path Symmetry-Aware Transformer-based mirror detection Network (SATNet), which includes two novel modules: Symmetry-Aware Attention Module (SAAM) and Contrast and Fusion Decoder Module (CFDM). Specifically, we first adopt a transformer backbone to model global information aggregation in images, extracting multi-scale features in two paths. We then feed the high-level dual-path features to SAAMs to capture the symmetry relations. Finally, we fuse the dual-path features and refine our prediction maps progressively with CFDMs to obtain the final mirror mask. Experimental results show that SATNet outperforms both RGB and RGB-D mirror detection methods on all available mirror detection datasets. Codes and trained models are available at: https://github.com/tyhuang0428/SATNet.
Code (1)
Tasks
DecoderMirror DetectionSimilar Papers 제목 키워드 기반
Mirror Symmetry Histograms for Capturing Geometric Properties in Images
We propose a data structure that captures global geometric properties in images: Histogram of Mirror Symmetry Coefficients. We compute such a coefficient for every pair of pixels, and group them in a 6-dimensional histog…
General ClassificationSymmetry DetectionMirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning
Most self-supervised learning (SSL) methods encourage invariance across augmentations, but strict flip invariance can suppress informative left--right correspondences in approximately bilateral data such as medical image…
Self-Supervised LearningRepresentation LearningFinding Mirror Symmetry via Registration
Symmetry is prevalent in nature and a common theme in man-made designs. Both the human visual system and computer vision algorithms can use symmetry to facilitate object recognition and other tasks. Detecting mirror symm…
Line DetectionObject RecognitionSymmetry Defense Against CNN Adversarial Perturbation Attacks
This paper uses symmetry to make Convolutional Neural Network classifiers (CNNs) robust against adversarial perturbation attacks. Such attacks add perturbation to original images to generate adversarial images that fool …
Adversarial RobustnessAutonomous VehiclesClassificationFlopping for FLOPs: Leveraging equivariance for computational efficiency
Incorporating geometric invariance into neural networks enhances parameter efficiency but typically increases computational costs. This paper introduces new equivariant neural networks that preserve symmetry while mainta…
Computational Efficiency