Training CNNs with Selective Allocation of Channels
Recent progress in deep convolutional neural networks (CNNs) have enabled a simple paradigm of architecture design: larger models typically achieve better accuracy. Due to this, in modern CNN architectures, it becomes more important to design models that generalize well under certain resource constraints, e.g. the number of parameters. In this paper, we propose a simple way to improve the capacity of any CNN model having large-scale features, without adding more parameters. In particular, we modify a standard convolutional layer to have a new functionality of channel-selectivity, so that the layer is trained to select important channels to re-distribute their parameters. Our experimental results under various CNN architectures and datasets demonstrate that the proposed new convolutional layer allows new optima that generalize better via efficient resource utilization, compared to the baseline.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Selective Convolutional Units: Improving CNNs via Channel Selectivity
Bottleneck structures with identity (e.g., residual) connection are now emerging popular paradigms for designing deep convolutional neural networks (CNN), for processing large-scale features efficiently. In this paper, w…
Model CompressionCNN Mixture-of-Depths
We introduce Mixture-of-Depths (MoD) for Convolutional Neural Networks (CNNs), a novel approach that enhances the computational efficiency of CNNs by selectively processing channels based on their relevance to the curren…
Computational EfficiencyCPUGPUSoft Masking for Cost-Constrained Channel Pruning
Structured channel pruning has been shown to significantly accelerate inference time for convolution neural networks (CNNs) on modern hardware, with a relatively minor loss of network accuracy. Recent works permanently z…
Linear Precoding Design for OTFS Systems in Time/Frequency Selective Fading Channels
Even orthogonal time frequency space (OTFS) has been shown as a promising modulation scheme for high mobility doubly-selective fading channels, its attainability of full diversity order in either time or frequency select…
DiversityChannel Locality Block: A Variant of Squeeze-and-Excitation
Attention mechanism is a hot spot in deep learning field. Using channel attention model is an effective method for improving the performance of the convolutional neural network. Squeeze-and-Excitation block takes advanta…