Minimax Bounds for Generalized Linear Models
We establish a new class of minimax prediction error bounds for generalized linear models. Our bounds significantly improve previous results when the design matrix is poorly structured, including natural cases where the matrix is wide or does not have full column rank. Apart from the typical $L_2$ risks, we study a class of entropic risks which recovers the usual $L_2$ prediction and estimation risks, and demonstrate that a tight analysis of Fisher information can uncover underlying structural dependency in terms of the spectrum of the design matrix. The minimax approach we take differs from the traditional metric entropy approach, and can be applied to many other settings.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionSimilar Papers 제목 키워드 기반
The Cost of Privacy in Generalized Linear Models: Algorithms and Minimax Lower Bounds
We propose differentially private algorithms for parameter estimation in both low-dimensional and high-dimensional sparse generalized linear models (GLMs) by constructing private versions of projected gradient descent. W…
LEMMAparameter estimationAdaptive deep learning for nonlinear time series models
In this paper, we develop a general theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs). We first consider two types of …
Deep LearningregressionTime SeriesTime Series Analysis+1Structured Low-Rank Tensors for Generalized Linear Models
Recent works have shown that imposing tensor structures on the coefficient tensor in regression problems can lead to more reliable parameter estimation and lower sample complexity compared to vector-based methods. This w…
parameter estimationregressionTensor DecompositionKullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models
Aggregation methods have emerged as a powerful and flexible framework in statistical learning, providing unified solutions across diverse problems such as regression, classification, and density estimation. In the contex…
Density EstimationA Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality
We study the problem of transfer learning, observing that previous efforts to understand its information-theoretic limits do not fully exploit the geometric structure of the source and target domains. In contrast, our st…
Transfer Learning