On Least Squares Estimation under Heteroscedastic and Heavy-Tailed Errors
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly bounded conditional variance and have only finitely many moments. We show that the interplay between the moment assumptions on the error, the metric entropy of the class of functions involved, and the "local" structure of the function class around the truth drives the rate of convergence of the LSE. We find sufficient conditions on the errors under which the rate of the LSE matches the rate of the LSE under sub-Gaussian error. Our results are finite sample and allow for heteroscedastic and heavy-tailed errors.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionSimilar Papers 제목 키워드 기반
Generalized Automatic Least Squares: Efficiency Gains from Misspecified Heteroscedasticity Models
It is well known that in the presence of heteroscedasticity ordinary least squares estimator is not efficient. I propose a generalized automatic least squares estimator (GALS) that makes partial correction of heterosceda…
A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise
We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a …
regressionLoss minimization and parameter estimation with heavy tails
This work studies applications and generalizations of a simple estimation technique that provides exponential concentration under heavy-tailed distributions, assuming only bounded low-order moments. We show that the tech…
parameter estimationregressionWeighted asymmetric least squares regression with fixed-effects
The fixed-effects model estimates the regressor effects on the mean of the response, which is inadequate to summarize the variable relationships in the presence of heteroscedasticity. In this paper, we adapt the asymmetr…
regressionOn Consistency and Asymptotic Normality of Least Absolute Deviation Estimators for 2-dimensional Sinusoidal Model
Estimation of the parameters of a 2-dimensional sinusoidal model is a fundamental problem in digital signal processing and time series analysis. In this paper, we propose a robust least absolute deviation (LAD) estimator…
parameter estimationTime SeriesTime Series Analysis