Learning $3$D-FilterMap for Deep Convolutional Neural Networks
We present a novel and compact architecture for deep Convolutional Neural Networks (CNNs) in this paper, termed $3$D-FilterMap Convolutional Neural Networks ($3$D-FM-CNNs). The convolution layer of $3$D-FM-CNN learns a compact representation of the filters, named $3$D-FilterMap, instead of a set of independent filters in the conventional convolution layer. The filters are extracted from the $3$D-FilterMap as overlapping $3$D submatrics with weight sharing among nearby filters, and these filters are convolved with the input to generate the output of the convolution layer for $3$D-FM-CNN. Due to the weight sharing scheme, the parameter size of the $3$D-FilterMap is much smaller than that of the filters to be learned in the conventional convolution layer when $3$D-FilterMap generates the same number of filters. Our work is fundamentally different from the network compression literature that reduces the size of a learned large network in the sense that a small network is directly learned from scratch. Experimental results demonstrate that $3$D-FM-CNN enjoys a small parameter space by learning compact $3$D-FilterMaps, while achieving performance compared to that of the baseline CNNs which learn the same number of filters as that generated by the corresponding $3$D-FilterMap.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Deep Analysis Dictionaries -- Part II: Convolutional Dictionaries
In this paper, we introduce a Deep Convolutional Analysis Dictionary Model (DeepCAM) by learning convolutional dictionaries instead of unstructured dictionaries as in the case of deep analysis dictionary model introduced…
ClusteringDictionary LearningImage Super-ResolutionSuper-ResolutionOn the Robustness of Convolutional Neural Networks to Internal Architecture and Weight Perturbations
Deep convolutional neural networks are generally regarded as robust function approximators. So far, this intuition is based on perturbations to external stimuli such as the images to be classified. Here we explore the ro…
A Dynamic Convolutional Layer for Short Range Weather Prediction
We present a new deep network layer called ``Dynamic Convolutional Layer" which is a generalization of the convolutional layer. The conventional convolutional layer uses filters that are learned during training and are h…
PredictionVC dimensions of group convolutional neural networks
We study the generalization capacity of group convolutional neural networks. We identify precise estimates for the VC dimensions of simple sets of group convolutional neural networks. In particular, we find that for infi…
A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification
Convolutional auto-encoders have shown their remarkable performance in stacking to deep convolutional neural networks for classifying image data during past several years. However, they are unable to construct the state-…
General Classificationimage-classificationImage Classification