Less Is More: A Comprehensive Framework for the Number of Components of Ensemble Classifiers
The number of component classifiers chosen for an ensemble greatly impacts the prediction ability. In this paper, we use a geometric framework for a priori determining the ensemble size, which is applicable to most of existing batch and online ensemble classifiers. There are only a limited number of studies on the ensemble size examining Majority Voting (MV) and Weighted Majority Voting (WMV). Almost all of them are designed for batch-mode, hardly addressing online environments. Big data dimensions and resource limitations, in terms of time and memory, make determination of ensemble size crucial, especially for online environments. For the MV aggregation rule, our framework proves that the more strong components we add to the ensemble, the more accurate predictions we can achieve. For the WMV aggregation rule, our framework proves the existence of an ideal number of components, which is equal to the number of class labels, with the premise that components are completely independent of each other and strong enough. While giving the exact definition for a strong and independent classifier in the context of an ensemble is a challenging task, our proposed geometric framework provides a theoretical explanation of diversity and its impact on the accuracy of predictions. We conduct a series of experimental evaluations to show the practical value of our theorems and existing challenges.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Compound Batch Normalization for Long-tailed Image Classification
Significant progress has been made in learning image classification neural networks under long-tail data distribution using robust training algorithms such as data re-sampling, re-weighting, and margin adjustment. Those …
Classificationimage-classificationImage ClassificationFrequency Regularization: Reducing Information Redundancy in Convolutional Neural Networks
Convolutional neural networks have demonstrated impressive results in many computer vision tasks. However, the increasing size of these networks raises concerns about the information overload resulting from the large num…
Supply Chain Forecasting in a Fast-Moving Global Economy: Review, Limits and Future Directions
The supply chain demand forecasting field has evolved to meet customer demand, preventing lost sales opportunities, and reducing maintenance costs. However, the large number of supply chain components (producers, vendors…
Demand ForecastingFrequency Regularization: Restricting Information Redundancy of Convolutional Neural Networks
Convolutional neural networks have demonstrated impressive results in many computer vision tasks. However, the increasing size of these networks raises concerns about the information overload resulting from the large num…
Iterative Thresholding for Demixing Structured Superpositions in High Dimensions
We consider the demixing problem of two (or more) high-dimensional vectors from nonlinear observations when the number of such observations is far less than the ambient dimension of the underlying vectors. Specifically, …
Vocal Bursts Intensity Prediction