Building Efficient Deep Neural Networks with Unitary Group Convolutions

2018-11-19 · CVPR 2019 6 · Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Christopher De Sa, Zhiru Zhang

We propose unitary group convolutions (UGConvs), a building block for CNNs which compose a group convolution with unitary transforms in feature space to learn a richer set of representations than group convolution alone. UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i.e. ShuffleNet) and block-circulant networks (i.e. CirCNN), and provide unifying insights that lead to a deeper understanding of each technique. We experimentally demonstrate that dense unitary transforms can outperform channel shuffling in DNN accuracy. On the other hand, different dense transforms exhibit comparable accuracy performance. Based on these observations we propose HadaNet, a UGConv network using Hadamard transforms. HadaNets achieve similar accuracy to circulant networks with lower computation complexity, and better accuracy than ShuffleNets with the same number of parameters and floating-point multiplies.

📄 PDF Abstract BibTeX arXiv:1811.07755

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Unitary convolutions for learning on graphs and groups

2024-10-07 · Bobak T. Kiani, Lukas Fesser, Melanie Weber

Data with geometric structure is ubiquitous in machine learning often arising from fundamental symmetries in a domain, such as permutation-invariance in graphs and translation-invariance in images. Group-convolutional ar…

Inductive Bias

Smoothness Errors in Dynamics Models and How to Avoid Them

2026-02-05 · Edward Berman, Luisa Li, Jung Yeon Park, Robin Walters arxiv

Modern neural networks have shown promise for solving partial differential equations over surfaces, often by discretizing the surface as a mesh and learning with a mesh-aware graph neural network. However, graph neural n…

Graph Neural Network Weather Forecasting

LieTransformer: Equivariant self-attention for Lie Groups

2020-12-20 · Michael Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont 외

Group equivariant neural networks are used as building blocks of group invariant neural networks, which have been shown to improve generalisation performance and data efficiency through principled parameter sharing. Such…

regression

Interleaved Group Convolutions

2017-10-01 · ICCV 2017 10 · Ting Zhang, Guo-Jun Qi, Bin Xiao, Jingdong Wang

In this paper, we present a simple and modularized neural network architecture, named interleaved group convolutional neural networks (IGCNets). The main point lies in a novel building block, a pair of two successive int…

Interleaved Group Convolutions for Deep Neural Networks

2017-07-10 · Ting Zhang, Guo-Jun Qi, Bin Xiao, Jingdong Wang