Can Evolutionary Sampling Improve Bagged Ensembles?
Perturb and Combine (P&C) group of methods generate multiple versions of the predictor by perturbing the training set or construction and then combining them into a single predictor (Breiman, 1996b). The motive is to improve the accuracy in unstable classification and regression methods. One of the most well known method in this group is Bagging. Arcing or Adaptive Resampling and Combining methods like AdaBoost are smarter variants of P&C methods. In this extended abstract, we lay the groundwork for a new family of methods under the P&C umbrella, known as Evolutionary Sampling (ES). We employ Evolutionary algorithms to suggest smarter sampling in both the feature space (sub-spaces) as well as training samples. We discuss multiple fitness functions to assess ensembles and empirically compare our performance against randomized sampling of training data and feature sub-spaces.
Code (1)
Tasks
Evolutionary AlgorithmsSimilar Papers 제목 키워드 기반
KNN Ensembles for Tweedie Regression: The Power of Multiscale Neighborhoods
Very few K-nearest-neighbor (KNN) ensembles exist, despite the efficacy of this approach in regression, classification, and outlier detection. Those that do exist focus on bagging features, rather than varying k or baggi…
Outlier DetectionregressionTopological Data AnalysisEvolutionary bagging for ensemble learning
Ensemble learning has gained success in machine learning with major advantages over other learning methods. Bagging is a prominent ensemble learning method that creates subgroups of data, known as bags, that are trained …
BIG-bench Machine LearningDiversityEnsemble LearningEvolutionary AlgorithmsBoosted Prompt Ensembles for Large Language Models
Methods such as chain-of-thought prompting and self-consistency have pushed the frontier of language model reasoning performance with no additional training. To further improve performance, we propose a prompt ensembling…
GSM8KLanguage ModelingLanguage ModellingSimplex-Constrained Sparse Bagging: Transitioning from Uniform Priors to Sparse Posteriors in Ensemble Learning
We present Simplex-Constrained Sparse Bagging (SCSB), a mathematically rigorous framework for post-training compression and probability calibration of bootstrap-based bagging ensembles. Standard bagging ensembles (such a…
Ensemble LearningScalable Subsampling Inference for Deep Neural Networks
Deep neural networks (DNN) has received increasing attention in machine learning applications in the last several years. Recently, a non-asymptotic error bound has been developed to measure the performance of the fully c…
Computational EfficiencyPredictionPrediction Intervalsvalid