Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators
Under the usual nonparametric regression model with Gaussian errors, Least Squares Estimators (LSEs) over natural subclasses of convex functions are shown to be suboptimal for estimating a $d$-dimensional convex function in squared error loss when the dimension $d$ is 5 or larger. The specific function classes considered include: (i) bounded convex functions supported on a polytope (in random design), (ii) Lipschitz convex functions supported on any convex domain (in random design), (iii) convex functions supported on a polytope (in fixed design). For each of these classes, the risk of the LSE is proved to be of the order $n^{-2/d}$ (up to logarithmic factors) while the minimax risk is $n^{-4/(d+4)}$, when $d \ge 5$. In addition, the first rate of convergence results (worst case and adaptive) for the unrestricted convex LSE are established in fixed-design for polytopal domains for all $d \geq 1$. Some new metric entropy results for convex functions are also proved which are of independent interest.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionSimilar Papers 제목 키워드 기반
On Suboptimality of Least Squares with Application to Estimation of Convex Bodies
We develop a technique for establishing lower bounds on the sample complexity of Least Squares (or, Empirical Risk Minimization) for large classes of functions. As an application, we settle an open problem regarding opti…
More Robust Estimators for Instrumental-Variable Panel Designs, With An Application to the Effect of Imports from China on US Employment
We show that first-difference two-stages-least-squares regressions identify non-convex combinations of location-and-period-specific treatment effects. Thus, those regressions could be biased if effects are heterogeneous.…
regressionStochastic Composite Least-Squares Regression with convergence rate O(1/n)
We consider the minimization of composite objective functions composed of the expectation of quadratic functions and an arbitrary convex function. We study the stochastic dual averaging algorithm with a constant step-siz…
regressionLeast-Squares Linear Dilation-Erosion Regressor Trained using a Convex-Concave Procedure
This paper presents a hybrid morphological neural network for regression tasks called linear dilation-erosion regressor ($\ell$-DER). An $\ell$-DER is given by a convex combination of the composition of linear and morpho…
regressionA Note On Nonlinear Regression Under L2 Loss
We investigate the nonlinear regression problem under L2 loss (square loss) functions. Traditional nonlinear regression models often result in non-convex optimization problems with respect to the parameter set. We show t…
regression