Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption
There has been a surge of interest in developing robust estimators for models with heavy-tailed and bounded variance data in statistics and machine learning, while few works impose unbounded variance. This paper proposes two type of robust estimators, the ridge log-truncated M-estimator and the elastic net log-truncated M-estimator. The first estimator is applied to convex regressions such as quantile regression and generalized linear models, while the other one is applied to high dimensional non-convex learning problems such as regressions via deep neural networks. Simulations and real data analysis demonstrate the {robustness} of log-truncated estimations over standard estimations.
Code (0)
등록된 구현이 없습니다.
Tasks
quantile regressionregressionSimilar Papers 제목 키워드 기반
Statistical Inference for Stochastic Gradient Descent Beyond Finite Variance
Stochastic gradient descent (SGD) is a foundational algorithm for large-scale statistical learning and stochastic optimization. However, statistical inference based on SGD iterates remains challenging when stochastic gra…
Stochastic OptimizationAsymptotic Theory for Random Forests
Random forests have proven to be reliable predictive algorithms in many application areas. Not much is known, however, about the statistical properties of random forests. Several authors have established conditions under…
Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximation
We study time-uniform statistical inference for parameters in stochastic approximation (SA), which encompasses a bunch of applications in optimization and machine learning. To that end, we analyze the almost-sure converg…
validAn Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
A Bayesian treatment can mitigate overconfidence in ReLU nets around the training data. But far away from them, ReLU Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be asymptotically overconf…
Multi-class ClassificationMarkov Chain Variance Estimation: A Stochastic Approximation Approach
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requir…
Reinforcement Learning (RL)