Harmonic Networks: Deep Translation and Rotation Equivariance

2016-12-14 · CVPR 2017 7 · Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, Gabriel J. Brostow

Translating or rotating an input image should not affect the results of many computer vision tasks. Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations. Global rotation equivariance is typically sought through data augmentation, but patch-wise equivariance is more difficult. We present Harmonic Networks or H-Nets, a CNN exhibiting equivariance to patch-wise translation and 360-rotation. We achieve this by replacing regular CNN filters with circular harmonics, returning a maximal response and orientation for every receptive field patch. H-Nets use a rich, parameter-efficient and low computational complexity representation, and we show that deep feature maps within the network encode complicated rotational invariants. We demonstrate that our layers are general enough to be used in conjunction with the latest architectures and techniques, such as deep supervision and batch normalization. We also achieve state-of-the-art classification on rotated-MNIST, and competitive results on other benchmark challenges.

📄 PDF Abstract BibTeX arXiv:1612.04642

Code (1)

deworrall92/harmonicConvolutions tf

Tasks

Data Augmentation Rotated MNIST Translation

Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance

2024-11-06 · Tomáš Karella, Adam Harmanec, Jan Kotera, Jan Blažek 외

CNNs exhibit inherent equivariance to image translation, leading to efficient parameter and data usage, faster learning, and improved robustness. The concept of translation equivariant networks has been successfully exte…

Translation

Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds

2018-02-22 · Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang 외

We introduce tensor field neural networks, which are locally equivariant to 3D rotations, translations, and permutations of points at every layer. 3D rotation equivariance removes the need for data augmentation to identi…

Data Augmentation Translation

Unified Spherical Frontend: Learning Rotation-Equivariant Representations of Spherical Images from Any Camera

2025-11-22 · Mukai Yu, Mosam Dabhi, Liuyue Xie, Sebastian Scherer 외 arxiv

Modern perception increasingly relies on fisheye, panoramic, and other wide field-of-view (FoV) cameras, yet most pipelines still apply planar CNNs designed for pinhole imagery on 2D grids, where pixel-space neighborhood…

Zero-shot Generalization

Spatial Attention Kinetic Networks with E(n)-Equivariance

2023-01-21 · Yuanqing Wang, John D. Chodera

Neural networks that are equivariant to rotations, translations, reflections, and permutations on n-dimensional geometric space have shown promise in physical modeling for tasks such as accurately but inexpensively model…

Robustness and Equivariance of Neural Networks

2019-05-01 · ICLR 2019 5 · Amit Deshpande, Sandesh Kamath, K V Subrahmanyam

Neural networks models are known to be vulnerable to geometric transformations as well as small pixel-wise perturbations of input. Convolutional Neural Networks (CNNs) are translation-equivariant but can be easily fooled…