Low-Cost Self-Ensembles Based on Multi-Branch Transformation and Grouped Convolution
Recent advancements in low-cost ensemble learning have demonstrated improved efficiency for image classification. However, the existing low-cost ensemble methods show relatively lower accuracy compared to conventional ensemble learning. In this paper, we propose a new low-cost ensemble learning, which can simultaneously achieve high efficiency and classification performance. A CNN is transformed into a multi-branch structure without introduction of additional components, which maintains the computational complexity as that of the original single model and also enhances diversity among the branches' outputs via sufficient separation between different pathways of the branches. In addition, we propose a new strategy that applies grouped convolution in the branches with different numbers of groups in different branches, which boosts the diversity of the branches' outputs. For training, we employ knowledge distillation using the ensemble of the outputs as the teacher signal. The high diversity among the outputs enables to form a powerful teacher, enhancing the individual branch's classification performance and consequently the overall ensemble performance. Experimental results show that our method achieves state-of-the-art classification accuracy and higher uncertainty estimation performance compared to previous low-cost ensemble methods. The code is available at https://github.com/hjdw2/SEMBG.
Code (1)
Tasks
ClassificationDiversityEnsemble Learningimage-classificationImage ClassificationKnowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unwarping Screen Content Images via Structure-texture Enhancement Network and Transformation Self-estimation
While existing implicit neural network-based image unwarping methods perform well on natural images, they struggle to handle screen content images (SCIs), which often contain large geometric distortions, text, symbols, a…
Deep interpretable ensembles
Ensembles improve prediction performance and allow uncertainty quantification by aggregating predictions from multiple models. In deep ensembling, the individual models are usually black box neural networks, or recently,…
Uncertainty QuantificationMulti-Narrow Transformation as a Single-Model Ensemble: Boundary Conditions, Mechanisms, and Failure Modes
Single-model ensembles (SMEs) have attracted attention as a way to approximate some of the benefits of deep ensembles within a single network. However, under an approximately matched parameter budget, it remains unclear …
Coupled Ensembles of Neural Networks
We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the globa…
Network Fission Ensembles for Low-Cost Self-Ensembles
Recent ensemble learning methods for image classification have been shown to improve classification accuracy with low extra cost. However, they still require multiple trained models for ensemble inference, which eventual…
Ensemble Learningimage-classificationImage Classification