Ensemble learning in CNN augmented with fully connected subnetworks
Convolutional Neural Networks (CNNs) have shown remarkable performance in general object recognition tasks. In this paper, we propose a new model called EnsNet which is composed of one base CNN and multiple Fully Connected SubNetworks (FCSNs). In this model, the set of feature-maps generated by the last convolutional layer in the base CNN is divided along channels into disjoint subsets, and these subsets are assigned to the FCSNs. Each of the FCSNs is trained independent of others so that it can predict the class label from the subset of the feature-maps assigned to it. The output of the overall model is determined by majority vote of the base CNN and the FCSNs. Experimental results using the MNIST, Fashion-MNIST and CIFAR-10 datasets show that the proposed approach further improves the performance of CNNs. In particular, an EnsNet achieves a state-of-the-art error rate of 0.16% on MNIST.
Code (2)
Tasks
Ensemble LearningImage ClassificationObject RecognitionSimilar Papers 제목 키워드 기반
A Combinatorial Theory of Dropout: Subnetworks, Graph Geometry, and Generalization
We propose a combinatorial and graph-theoretic theory of dropout by modeling training as a random walk over a high-dimensional graph of binary subnetworks. Each node represents a masked version of the network, and dropou…
Learning on a Razor's Edge: the Singularity Bias of Polynomial Neural Networks
Deep neural networks often infer sparse representations, converging to a subnetwork during the learning process. In this work, we theoretically analyze subnetworks and their bias through the lens of algebraic geometry. W…
Polynomial Neural NetworksMixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks
Recent strategies achieved ensembling "for free" by fitting concurrently diverse subnetworks inside a single base network. The main idea during training is that each subnetwork learns to classify only one of the multiple…
image-classificationImage ClassificationLottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
Lottery tickets (LTs) is able to discover accurate and sparse subnetworks that could be trained in isolation to match the performance of dense networks. Ensemble, in parallel, is one of the oldest time-proven tricks in m…
Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model
Ensembling is a popular method used to improve performance as a last resort. However, ensembling multiple models finetuned from a single pretrained model has been not very effective; this could be due to the lack of dive…
Diversity