Extending the Use of MDL for High-Dimensional Problems: Variable Selection, Robust Fitting, and Additive Modeling
In the signal processing and statistics literature, the minimum description length (MDL) principle is a popular tool for choosing model complexity. Successful examples include signal denoising and variable selection in linear regression, for which the corresponding MDL solutions often enjoy consistent properties and produce very promising empirical results. This paper demonstrates that MDL can be extended naturally to the high-dimensional setting, where the number of predictors $p$ is larger than the number of observations $n$. It first considers the case of linear regression, then allows for outliers in the data, and lastly extends to the robust fitting of nonparametric additive models. Results from numerical experiments are presented to demonstrate the efficiency and effectiveness of the MDL approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Additive modelsDenoisingregressionVariable SelectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Differentially Private High-dimensional Variable Selection via Integer Programming
Sparse variable selection improves interpretability and generalization in high-dimensional learning by selecting a small subset of informative features. Recent advances in Mixed Integer Programming (MIP) have enabled sol…
Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization
Bayesian optimization (BO) is a class of popular methods for expensive black-box optimization, and has been widely applied to many scenarios. However, BO suffers from the curse of dimensionality, and scaling it to high-d…
Bayesian OptimizationMuJoCoVariable SelectionHigh Dimensional Bayesian Optimization using Lasso Variable Selection
Bayesian optimization (BO) is a leading method for optimizing expensive black-box optimization and has been successfully applied across various scenarios. However, BO suffers from the curse of dimensionality, making it c…
Bayesian OptimizationComputational EfficiencyVariable SelectionFeature selection in functional data classification with recursive maxima hunting
Dimensionality reduction is one of the key issues in the design of effective machine learning methods for automatic induction. In this work, we introduce recursive maxima hunting (RMH) for variable selection in classific…
Dimensionality Reductionfeature selectionGeneral ClassificationVariable SelectionAccurate Estimation of Quantitative Trait Locus Effects with Epistatic by Improved Variational Linear Regression
Bayesian approaches to variable selection have been widely used for quantitative trait locus (QTL) mapping. The Markov chain Monte Carlo (MCMC) algorithms for that aim are often difficult to be implemented for high-dimen…
regressionVariable Selection