Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning
This paper studies the performance of deep convolutional neural networks (DCNNs) with zero-padding in feature extraction and learning. After verifying the roles of zero-padding in enabling translation-equivalence, and pooling in its translation-invariance driven nature, we show that with similar number of free parameters, any deep fully connected networks (DFCNs) can be represented by DCNNs with zero-padding. This demonstrates that DCNNs with zero-padding is essentially better than DFCNs in feature extraction. Consequently, we derive universal consistency of DCNNs with zero-padding and show its translation-invariance in the learning process. All our theoretical results are verified by numerical experiments including both toy simulations and real-data running.
Code (1)
Tasks
TranslationSimilar Papers 제목 키워드 기반
Improving Translation Invariance in Convolutional Neural Networks with Peripheral Prediction Padding
Zero padding is often used in convolutional neural networks to prevent the feature map size from decreasing with each layer. However, recent studies have shown that zero padding promotes encoding of absolute positional i…
Semantic SegmentationTranslationContext-aware Padding for Semantic Segmentation
Zero padding is widely used in convolutional neural networks to prevent the size of feature maps diminishing too fast. However, it has been claimed to disturb the statistics at the border. As an alternative, we propose a…
SegmentationSemantic SegmentationOn the Interplay of Convolutional Padding and Adversarial Robustness
It is common practice to apply padding prior to convolution operations to preserve the resolution of feature-maps in Convolutional Neural Networks (CNN). While many alternatives exist, this is often achieved by adding a …
Adversarial RobustnessInterpolation-Aware Padding for 3D Sparse Convolutional Neural Networks
Sparse voxel-based 3D convolutional neural networks (CNNs) are widely used for various 3D vision tasks. Sparse voxel-based 3D CNNs create sparse non-empty voxels from the 3D input and perform 3D convolution operations on…
SegmentationSemantic SegmentationSpectral Norm of Convolutional Layers with Circular and Zero Paddings
This paper leverages the use of \emph{Gram iteration} an efficient, deterministic, and differentiable method for computing spectral norm with an upper bound guarantee. Designed for circular convolutional layers, we gener…