Steerable Transformers

2024-05-24 · Soumyabrata Kundu, Risi Kondor

In this work we introduce Steerable Transformers, an extension of the Vision Transformer mechanism that maintains equivariance to the special Euclidean group $\mathrm{SE}(d)$. We propose an equivariant attention mechanism that operates on features extracted by steerable convolutions. Operating in Fourier space, our network utilizes Fourier space non-linearities. Our experiments in both two and three dimensions show that adding a steerable transformer encoder layer to a steerable convolution network enhances performance.

📄 PDF Abstract BibTeX arXiv:2405.15932

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Attention 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Adam 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks

2019-06-10 · Rohan Ghosh, Anupam K. Gupta

Augmenting transformation knowledge onto a convolutional neural network's weights has often yielded significant improvements in performance. For rotational transformation augmentation, an important element to recent appr…

Steerable Visual Representations

2026-04-02 · Jona Ruthardt, Manu Gaur, Deva Ramanan, Makarand Tapaswi 외 arxiv

Pretrained Vision Transformers (ViTs) such as DINOv2 and MAE provide generic image features that can be applied to a variety of downstream tasks such as retrieval, classification, and segmentation. However, such represen…

Zero-shot Generalization Anomaly Detection

RoTIR: Rotation-Equivariant Network and Transformers for Fish Scale Image Registration

2024-01-20 · Ruixiong Wang, Alin Achim, Renata Raele-Rolfe, Qiao Tong 외

Image registration is an essential process for aligning features of interest from multiple images. With the recent development of deep learning techniques, image registration approaches have advanced to a new level. In t…

Deep Learning Image Registration

Steerable CNNs

2016-12-27 · Taco S. Cohen, Max Welling

It has long been recognized that the invariance and equivariance properties of a representation are critically important for success in many vision tasks. In this paper we present Steerable Convolutional Neural Networks,…

General Classification image-classification Image Classification

FILTRA: Rethinking Steerable CNN by Filter Transform

2021-05-25 · Bo Li, Qili Wang, Gim Hee Lee

Steerable CNN imposes the prior knowledge of transformation invariance or equivariance in the network architecture to enhance the the network robustness on geometry transformation of data and reduce overfitting. It has b…