Interpolating Convolutional Neural Networks Using Batch Normalization
Perceiving a visual concept as a mixture of learned ones is natural for humans, aiding them to grasp new concepts and strengthening old ones. For all their power and recent success, deep convolutional networks do not have this ability. Inspired by recent work on universal representations for neural networks, we propose a simple emulation of this mechanism by purposing batch normalization layers to discriminate visual classes, and formulating a way to combine them to solve new tasks. We show that this can be applied for 2-way few-shot learning where we obtain between 4% and 17% better accuracy compared to straightforward full fine-tuning, and demonstrate that it can also be extended to the orthogonal application of style transfer.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningStyle TransferMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Kernel Normalized Convolutional Networks
Existing convolutional neural network architectures frequently rely upon batch normalization (BatchNorm) to effectively train the model. BatchNorm, however, performs poorly with small batch sizes, and is inapplicable to …
Federated Learningimage-classificationImage ClassificationSemantic SegmentationThe Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks
We study the implicit bias of batch normalization trained by gradient descent. We show that when learning a linear model with batch normalization for binary classification, gradient descent converges to a uniform margin …
Binary ClassificationVariance-Aware Weight Initialization for Point Convolutional Neural Networks
Appropriate weight initialization has been of key importance to successfully train neural networks. Recently, batch normalization has diminished the role of weight initialization by simply normalizing each layer based on…
Batch Layer Normalization, A new normalization layer for CNNs and RNN
This study introduces a new normalization layer termed Batch Layer Normalization (BLN) to reduce the problem of internal covariate shift in deep neural network layers. As a combined version of batch and layer normalizati…
Comparing Normalization Methods for Limited Batch Size Segmentation Neural Networks
The widespread use of Batch Normalization has enabled training deeper neural networks with more stable and faster results. However, the Batch Normalization works best using large batch size during training and as the sta…
Segmentation