Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization

2021-02-12 · Christina Runkel, Christian Etmann, Michael Möller, Carola-Bibiane Schönlieb

An increasing number of models require the control of the spectral norm of convolutional layers of a neural network. While there is an abundance of methods for estimating and enforcing upper bounds on those during training, they are typically costly in either memory or time. In this work, we introduce a very simple method for spectral normalization of depthwise separable convolutions, which introduces negligible computational and memory overhead. We demonstrate the effectiveness of our method on image classification tasks using standard architectures like MobileNetV2.

📄 PDF Abstract BibTeX arXiv:2102.06496

Code (0)

등록된 구현이 없습니다.

Tasks

image-classification Image Classification

Methods 이 논문이 사용한 방법론

Depthwise Convolution Depthwise Convolution is a type of convolution where we apply a single convolutional filter for each input channel. In the regular 2D…

Pointwise Convolution Pointwise Convolution is a type of convolution that uses a 1x1 kernel: a kernel that iterates through every single point. This…

Batch Normalization 설명 없음

Depthwise Separable Convolution While standard convolution performs the channelwise and spatial-wise computation in one step, Depthwise Separable Convolution …

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Inverted Residual Block 설명 없음

Average Pooling 설명 없음

Accelerating Depthwise Separable Convolutions on Ultra-Low-Power Devices

2024-06-18 · Francesco Daghero, Alessio Burrello, Massimo Poncino, Enrico Macii 외

Depthwise separable convolutions are a fundamental component in efficient Deep Neural Networks, as they reduce the number of parameters and operations compared to traditional convolutions while maintaining comparable acc…

Network Decoupling: From Regular to Depthwise Separable Convolutions

2018-08-16 · Jianbo Guo, Yuxi Li, Weiyao Lin, Yurong Chen 외

Depthwise separable convolution has shown great efficiency in network design, but requires time-consuming training procedure with full training-set available. This paper first analyzes the mathematical relationship betwe…

object-detection Object Detection

QuickNet: Maximizing Efficiency and Efficacy in Deep Architectures

2017-01-09 · Tapabrata Ghosh

We present QuickNet, a fast and accurate network architecture that is both faster and significantly more accurate than other fast deep architectures like SqueezeNet. Furthermore, it uses less parameters than previous net…

Model Compression

3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks

2018-08-05 · Rongtian Ye, Fangyu Liu, Liqiang Zhang

Standard 3D convolution operations require much larger amounts of memory and computation cost than 2D convolution operations. The fact has hindered the development of deep neural nets in many 3D vision tasks. In this pap…

General Classification

Depthwise Separable Convolutions for Neural Machine Translation

2017-06-09 · ICLR 2018 1 · Lukasz Kaiser, Aidan N. Gomez, Francois Chollet

Depthwise separable convolutions reduce the number of parameters and computation used in convolutional operations while increasing representational efficiency. They have been shown to be successful in image classificatio…

image-classification Machine Translation Translation