Beyond Maximum Likelihood: Variational Inequality Estimation for Generalized Linear Models
Generalized linear models (GLMs) are fundamental tools for statistical modeling, with maximum likelihood estimation (MLE) serving as the classical approach for parameter inference. While MLE performs well for canonical GLMs, it can become computationally challenging in more general settings with non-canonical, non-smooth, or nonlinear link functions, where the resulting optimization landscape may be ill-conditioned, non-convex, or non-differentiable. In this paper, we study an alternative estimation framework based on variational inequalities (VIs), which formulates GLM estimation through an operator-based equilibrium condition rather than likelihood minimization. We analyze the VI estimator from a statistical perspective and establish finite-sample error bounds and asymptotic normality under mild regularity conditions, together with convergence guarantees for fixed-point and stochastic approximation algorithms. The framework accommodates a broad class of link functions, including non-canonical and non-monotone cases satisfying a strong Minty-type condition, and extends naturally to generalized additive models via basis expansion. Numerical experiments demonstrate that the VI approach achieves competitive finite-sample accuracy and improved numerical stability relative to MLE, particularly in GLMs and GAMs with non-canonical or non-smooth link functions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Majorization for CRFs and Latent Likelihoods
The partition function plays a key role in probabilistic modeling including conditional random fields, graphical models, and maximum likelihood estimation. To optimize partition functions, this article introduces a quadr…
Finite-sample risk bounds for maximum likelihood estimation with arbitrary penalties
The MDL two-part coding $ \textit{index of resolvability} $ provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models. However, the bound does not apply to unpe…
Variational Approximated Restricted Maximum Likelihood Estimation for Spatial Data
This research considers a scalable inference for spatial data modeled through Gaussian intrinsic conditional autoregressive (ICAR) structures. The classical estimation method, restricted maximum likelihood (REML), requir…
Variational inequality for perpetual American option price and convergence to the solution of the difference equation
A variational inequality for pricing the perpetual American option and the corresponding difference equation are considered. First, the maximum principle and uniqueness of the solution to variational inequality for prici…
Nonnegative dictionary learning in the exponential noise model for adaptive music signal representation
In this paper we describe a maximum likelihood likelihood approach for dictionary learning in the multiplicative exponential noise model. This model is prevalent in audio signal processing where it underlies a generative…
Audio Signal ProcessingDictionary Learning