Reducing Sampling Ratios Improves Bagging in Sparse Regression

2018-12-20 · Luoluo Liu, Sang Peter Chin, Trac. D. Tran

Bagging, a powerful ensemble method from machine learning, improves the performance of unstable predictors. Although the power of Bagging has been shown mostly in classification problems, we demonstrate the success of employing Bagging in sparse regression over the baseline method (L1 minimization). The framework employs the generalized version of the original Bagging with various bootstrap ratios. The performance limits associated with different choices of bootstrap sampling ratio L/m and number of estimates K is analyzed theoretically. Simulation shows that the proposed method yields state-of-the-art recovery performance, outperforming L1 minimization and Bolasso in the challenging case of low levels of measurements. A lower L/m ratio (60% - 90%) leads to better performance, especially with a small number of measurements. With the reduced sampling rate, SNR improves over the original Bagging by up to 24%. With a properly chosen sampling ratio, a reasonably small number of estimates K = 30 gives satisfying result, even though increasing K is discovered to always improve or at least maintain the performance.

📄 PDF Abstract BibTeX arXiv:1812.08808

Code (0)

등록된 구현이 없습니다.

Tasks

regression

Exact Finite-Sample Variance Decomposition of Subagging: A Spectral Filtering Perspective

2026-04-12 · Ye Su, Mingrui Ye, Yining Wang, Jipeng Guo 외 arxiv

Standard resampling ratios (e.g., $α\approx 0.632$) are widely used as default baselines in ensemble learning for three decades. However, how these ratios interact with a base learner's intrinsic functional complexity in…

Ensemble Learning

Feature Bagging Provides Stability

2026-07-29 · Yuheng Ma, Qiang Sun arxiv

We study feature bagging through the lens of algorithmic stability. Feature bagging is an ensemble strategy that aggregates base learners trained on randomly subsampled feature subsets, possibly in a data-dependent manne…

Active-SAOOD: Active Sparsely Annotated Oriented Object Detection in Remote Sensing Images

2026-05-11 · Yu Lin, Jianghang Lin, Kai Ye, Shengchuan Zhang 외 arxiv

Reducing the annotation cost of oriented object detection in remote sensing remains a major challenge. Recently, sparse annotation has gained attention for effectively reducing annotation redundancy in densely remote sen…

Object Detection Active Learning

Stacking and stability

2019-01-26 · Nino Arsov, Martin Pavlovski, Ljupco Kocarev

Stacking is a general approach for combining multiple models toward greater predictive accuracy. It has found various application across different domains, ensuing from its meta-learning nature. Our understanding, nevert…

Meta-Learning

Augmented Physics-Based Li-ion Battery Model via Adaptive Ensemble Sparse Learning and Conformal Prediction

2025-07-01 · Samuel Filgueira da Silva, Mehmet Fatih Ozkan, Faissal El Idrissi, Marcello Canova arxiv

Accurate electrochemical models are essential for the safe and efficient operation of lithium-ion batteries in real-world applications such as electrified vehicles and grid storage. Reduced-order models (ROM) offer a bal…

Computational Efficiency Sparse Learning