Bayesian Imbalanced Regression Debiasing
Imbalanced regression, where the training data has an uneven distribution on its range, is widely encountered in the real world, e.g., age estimation (uni-dimensional regression) and pose estimation (multi-dimensional regression). Compared to imbalanced and long-tailed classification, imbalanced regression has its unique challenges as the regression label space can be continuous, boundless, and high-dimensional. In this work, we present a principled framework, Bayesian Posterior Debiasing (Bayesian-PD), for re-balancing the regression among frequent and rare observations. Our key insight is that a balanced posterior can be obtained by debiasing the conditional probability with a regression label space prior. Importantly, through a normalization reparameterization technique, we derive a general debiasing function between the empirical posterior and the balanced posterior without relying on task-specific assumptions. We show that the Bayesian-PD framework has multiple instantiations in both training and testing time, with either closed-form or numerical implementations. We further uncover that several existing methods in imbalanced classification/regression serve as special cases of our Bayesian-PD framework. Extensive experiments on both uni- and multi-dimensional regression benchmarks demonstrate the effectiveness of the Bayesian-PD framework on various real-world tasks. Notably, Bayesian-PD exhibits strong robustness to different skewness of the training distributions.
Code (0)
등록된 구현이 없습니다.
Tasks
Age Estimationimbalanced classificationPose EstimationregressionSimilar Papers 제목 키워드 기반
Fair for a few: Improving Fairness in Doubly Imbalanced Datasets
Fairness has been identified as an important aspect of Machine Learning and Artificial Intelligence solutions for decision making. Recent literature offers a variety of approaches for debiasing, however many of them fall…
AttributeDecision MakingFairnessDebiased Bayesian Inference for High-dimensional Regression Models
There has been significant progress in Bayesian inference based on sparsity-inducing (e.g., spike-and-slab and horseshoe-type) priors for high-dimensional regression models. The resulting posteriors, however, in general …
Bayesian InferenceFully Bayesian Analysis of the Relevance Vector Machine Classification for Imbalanced Data
Relevance Vector Machine (RVM) is a supervised learning algorithm extended from Support Vector Machine (SVM) based on the Bayesian sparsity model. Compared with the regression problem, RVM classification is difficult to …
ClassificationGeneral ClassificationSpectrum-Aware Debiasing: A Modern Inference Framework with Applications to Principal Components Regression
Debiasing is a fundamental concept in high-dimensional statistics. While degrees-of-freedom adjustment is the state-of-the-art technique in high-dimensional linear regression, it is limited to i.i.d. samples and sub-Gaus…
compressed sensingregressionFast Mixing of Data Augmentation Algorithms: Bayesian Probit, Logit, and Lasso Regression
Despite the widespread use of the data augmentation (DA) algorithm, the theoretical understanding of its convergence behavior remains incomplete. We prove the first non-asymptotic polynomial upper bounds on mixing times …
Data Augmentationregression