paper-with-me

Papers

CondConv: Conditionally Parameterized Convolutions for Efficient Inference

2019-04-10 · NeurIPS 2019 12 · Brandon Yang, Gabriel Bender, Quoc V. Le, Jiquan Ngiam

Convolutional layers are one of the basic building blocks of modern deep neural networks. One fundamental assumption is that convolutional kernels should be shared for all examples in a dataset. We propose conditionally parameterized convolutions (CondConv), which learn specialized convolutional kernels for each example. Replacing normal convolutions with CondConv enables us to increase the size and capacity of a network, while maintaining efficient inference. We demonstrate that scaling networks with CondConv improves the performance and inference cost trade-off of several existing convolutional neural network architectures on both classification and detection tasks. On ImageNet classification, our CondConv approach applied to EfficientNet-B0 achieves state-of-the-art performance of 78.3% accuracy with only 413M multiply-adds. Code and checkpoints for the CondConv Tensorflow layer and CondConv-EfficientNet models are available at: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/condconv.

📄 PDF Abstract BibTeX arXiv:1904.04971

Code (9)

tensorflow/tpu 공식 구현 tf
hangg7/deformable-kernels pytorch
hangg7/deformable-kernels/ pytorch
hey-yahei/CondConv.MXNet mxnet
mindspore-courses/External-Attention-MindSpore/blob/main/model/conv/CondConv.py mindspore
prstrive/CondConv-tensorflow tf
rwightman/gen-efficientnet-pytorch pytorch
tensorflow/tpu/tree/master/models/official/efficientnet/condconv tf
xmu-xiaoma666/External-Attention-pytorch pytorch

Tasks

General ClassificationImage ClassificationObject Detection

Methods 이 논문이 사용한 방법론

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…
RMSProp RMSProp is an unpublished adaptive learning rate optimizer proposed by Geoff Hinton. The motivation…
Tanh Activation 설명 없음
Residual Connection 설명 없음
Sigmoid Activation 설명 없음
ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Depthwise Convolution Depthwise Convolution is a type of convolution where we apply a single convolutional filter for each input channel. In the regular 2D…

Similar Papers 제목 키워드 기반

EXTENDING CONDITIONAL CONVOLUTION STRUCTURES FOR ENHANCING MULTITASKING CONTINUAL LEARNING

2020-12-07 · Cheng-Hao Tu, Cheng-En Wu, Chu-Song Chen

Conditional operations have received much attention in recent deep learning studies to facilitate the prediction accuracy of a model. A recent advance toward this direction is the conditional parametric convolutions (Con…

Computational EfficiencyContinual LearningMulti-Task Learning

Learning to utilize image second-order derivative information for crisp edge detection

2024-06-09 · Changsong Liu, Yimeng Fan, Mingyang Li, Wei zhang 외

Edge detection is a fundamental task in computer vision. It has made great progress under the development of deep convolutional neural networks (DCNNs), some of which have achieved a beyond human-level performance. Howev…

Edge Detection

Extrapolating Phase-Field Simulations in Space and Time with Purely Convolutional Architectures

2025-09-25 · Christophe Bonneville, Nathan Bieberdorf, Pieterjan Robbe, Mark Asta 외 arxiv

Phase-field models of liquid metal dealloying (LMD) can resolve rich microstructural dynamics but become intractable for large domains or long time horizons. We present a conditionally parameterized, fully convolutional …

Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling

2024-08-18 · Harry Jake Cunningham, Giorgio Giannone, Mingtian Zhang, Marc Peter Deisenroth

Global convolutions have shown increasing promise as powerful general-purpose sequence models. However, training long convolutions is challenging, and kernel parameterizations must be able to learn long-range dependencie…

RTFormer: Re-parameter TSBN Spiking Transformer

2024-06-20 · Hongzhi Wang, Xiubo Liang, Mengjian Li, Tao Zhang

The Spiking Neural Networks (SNNs), renowned for their bio-inspired operational mechanism and energy efficiency, mirror the human brain's neural activity. Yet, SNNs face challenges in balancing energy efficiency with the…