Ensemble pruning via an integer programming approach with diversity constraints
Ensemble learning combines multiple classifiers in the hope of obtaining better predictive performance. Empirical studies have shown that ensemble pruning, that is, choosing an appropriate subset of the available classifiers, can lead to comparable or better predictions than using all classifiers. In this paper, we consider a binary classification problem and propose an integer programming (IP) approach for selecting optimal classifier subsets. We propose a flexible objective function to adapt to desired criteria of different datasets. We also propose constraints to ensure minimum diversity levels in the ensemble. Despite the general case of IP being NP-Hard, state-of-the-art solvers are able to quickly obtain good solutions for datasets with up to 60000 data points. Our approach yields competitive results when compared to some of the best and most used pruning methods in literature.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationDiversityEnsemble LearningEnsemble PruningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optimally Pruning Decision Tree Ensembles With Feature Cost
We consider the problem of learning decision rules for prediction with feature budget constraint. In particular, we are interested in pruning an ensemble of decision trees to reduce expected feature cost while maintainin…
Ensemble PruningPredictionPruning Techniques for Mixed Ensembles of Genetic Programming Models
The objective of this paper is to define an effective strategy for building an ensemble of Genetic Programming (GP) models. Ensemble methods are widely used in machine learning due to their features: they average out bia…
DiversityHierarchical Pruning of Deep Ensembles with Focal Diversity
Deep neural network ensembles combine the wisdom of multiple deep neural networks to improve the generalizability and robustness over individual networks. It has gained increasing popularity to study deep ensemble techni…
Decision MakingDiversityEnsemble PruningDeep Ensembles with Hierarchical Diversity Pruning
Diverse deep ensembles hold the potential for improving accuracy and robustness of deep learning models. Both pairwise and non-pairwise ensemble diversity metrics have been proposed over the past two decades. However, it…
Diversity