Adaptive Ensemble Prediction for Deep Neural Networks based on Confidence Level
Ensembling multiple predictions is a widely used technique for improving the accuracy of various machine learning tasks. One obvious drawback of ensembling is its higher execution cost during inference. In this paper, we first describe our insights on the relationship between the probability of prediction and the effect of ensembling with current deep neural networks; ensembling does not help mispredictions for inputs predicted with a high probability even when there is a non-negligible number of mispredicted inputs. This finding motivated us to develop a way to adaptively control the ensembling. If the prediction for an input reaches a high enough probability, i.e., the output from the softmax function, on the basis of the confidence level, we stop ensembling for this input to avoid wasting computation power. We evaluated the adaptive ensembling by using various datasets and showed that it reduces the computation cost significantly while achieving accuracy similar to that of static ensembling using a pre-defined number of local predictions. We also show that our statistically rigorous confidence-level-based early-exit condition reduces the burden of task-dependent threshold tuning better compared with naive early exit based on a pre-defined threshold in addition to yielding a better accuracy with the same cost.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Fast and Accurate Inference with Adaptive Ensemble Prediction for Deep Networks
Ensembling multiple predictions is a widely-used technique to improve the accuracy of various machine learning tasks. In image classification tasks, for example, averaging the predictions for multiple patches extracted f…
image-classificationImage ClassificationAdaptive Ensemble Learning with Confidence Bounds
Extracting actionable intelligence from distributed, heterogeneous, correlated and high-dimensional data sources requires run-time processing and learning both locally and globally. In the last decade, a large number of …
Ensemble LearningMeta-LearningSQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks
Early-exit neural networks have become popular for reducing inference latency by allowing intermediate predictions when sufficient confidence is achieved. However, standard approaches typically rely on single-model confi…
Neural Architecture SearchEnsemble LearningHarnessing Consistency for Robust Test-Time LLM Ensemble
Different large language models (LLMs) exhibit diverse strengths and weaknesses, and LLM ensemble serves as a promising approach to integrate their complementary capabilities. Despite substantial progress in improving en…
Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Errors for Deep Neural Networks
Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase th…
Conformal PredictionDeep LearningDrug DiscoveryPrediction