Developing parsimonious ensembles using predictor diversity within a reinforcement learning framework
Heterogeneous ensembles that can aggregate an unrestricted number and variety of base predictors can effectively address challenging prediction problems. In particular, accurate ensembles that are also parsimonious, i.e., consist of as few base predictors as possible, can help reveal potentially useful knowledge about the target problem domain. Although ensemble selection offers a potential approach to achieving these goals, the currently available algorithms are limited in their abilities. In this paper, we present several algorithms that incorporate ensemble diversity into a reinforcement learning (RL)-based ensemble selection framework to build accurate and parsimonious ensembles. These algorithms, as well as several baselines, are rigorously evaluated on datasets from diverse domains in terms of the predictive performance and parsimony of their ensembles. This evaluation demonstrates that our diversity-incorporated RL-based algorithms perform better than the others for constructing simultaneously accurate and parsimonious ensembles. These algorithms can eventually aid the interpretation or reverse engineering of predictive models assimilated into effective ensembles. To enable such a translation, an implementation of these algorithms, as well the experimental setup they are evaluated in, has been made available at https://github.com/GauravPandeyLab/lens-learning-ensembles-using-reinforcement-learning.
Code (1)
Tasks
Diversityreinforcement-learningReinforcement LearningReinforcement Learning (RL)TranslationSimilar Papers 제목 키워드 기반
Developing parsimonious ensembles using ensemble diversity within a reinforcement learning framework
Heterogeneous ensembles built from the predictions of a wide variety and large number of diverse base predictors represent a potent approach to building predictive models for problems where the ideal base/individual pred…
Diversityreinforcement-learningReinforcement LearningReinforcement Learning (RL)Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
Recent studies on deep ensembles have identified the sharpness of the local minima of individual learners and the diversity of the ensemble members as key factors in improving test-time performance. Building on this, our…
DiversityRe-purposing Heterogeneous Generative Ensembles with Evolutionary Computation
Generative Adversarial Networks (GANs) are popular tools for generative modeling. The dynamics of their adversarial learning give rise to convergence pathologies during training such as mode and discriminator collapse. I…
DiversityEvolutionary AlgorithmsWhen does Diversity Help Generalization in Classification Ensembles?
Ensembles, as a widely used and effective technique in the machine learning community, succeed within a key element -- "diversity." The relationship between diversity and generalization, unfortunately, is not entirely un…
ClassificationDiversityEnsemble PruningGeneral Classification+1Deep Ensembles: A Loss Landscape Perspective
Deep ensembles have been empirically shown to be a promising approach for improving accuracy, uncertainty and out-of-distribution robustness of deep learning models. While deep ensembles were theoretically motivated by t…