Normalized Convolutional Neural Network
In this paper, we propose Normalized Convolutional Neural Network(NCNN). NCNN is more fitted to a convolutional operator than other nomralizaiton methods. The normalized process is similar to a normalization methods, but NCNN is more adapative to sliced-inputs and corresponding the convolutional kernel. Therefor NCNN can be targeted to micro-batch training. Normalizaing of NC is conducted during convolutional process. In short, NC process is not usual normalization and can not be realized in deep learning framework optimizing standard convolution process. Hence we named this method 'Normalized Convolution'. As a result, NC process has universal property which means NC can be applied to any AI tasks involving convolution neural layer . Since NC don't need other normalization layer, NCNN looks like convolutional version of Self Normalizing Network.(SNN). Among micro-batch trainings, NCNN outperforms other batch-independent normalization methods. NCNN archives these superiority by standardizing rows of im2col matrix of inputs, which theoretically smooths the gradient of loss. The code need to manipulate standard convolution neural networks step by step. The code is available : https://github.com/kimdongsuk1/ NormalizedCNN.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification
We introduce globally normalized convolutional neural networks for joint entity classification and relation extraction. In particular, we propose a way to utilize a linear-chain conditional random field output layer for …
General ClassificationRelationRelation ClassificationRelation ExtractionKernel Normalized Convolutional Networks
Existing convolutional neural network architectures frequently rely upon batch normalization (BatchNorm) to effectively train the model. BatchNorm, however, performs poorly with small batch sizes, and is inapplicable to …
Federated Learningimage-classificationImage ClassificationSemantic SegmentationBird Species Categorization Using Pose Normalized Deep Convolutional Nets
We propose an architecture for fine-grained visual categorization that approaches expert human performance in the classification of bird species. Our architecture first computes an estimate of the object's pose; this is …
ClassificationClusteringFine-Grained Visual CategorizationGeneral Classification1 bit is all we need: binary normalized neural networks
The increasing size of large neural network models, specifically language models and foundational image models, poses deployment challenges, prompting efforts to reduce memory requirements and enhance computational effic…
Computational EfficiencyImage ClassificationNormalized Cut Loss for Weakly-supervised CNN Segmentation
Most recent semantic segmentation methods train deep convolutional neural networks with fully annotated masks requiring pixel-accuracy for good quality training. Common weakly-supervised approaches generate full masks fr…
Interactive SegmentationSegmentationSemantic Segmentation