On Learning-Curve Monotonicity for Maximum Likelihood Estimators
The property of learning-curve monotonicity, highlighted in a recent series of work by Loog, Mey and Viering, describes algorithms which only improve in average performance given more data, for any underlying data distribution within a given family. We establish the first nontrivial monotonicity guarantees for the maximum likelihood estimator in a variety of well-specified parametric settings. For sequential prediction with log loss, we show monotonicity (in fact complete monotonicity) of the forward KL divergence for Gaussian vectors with unknown covariance and either known or unknown mean, as well as for Gamma variables with unknown scale parameter. The Gaussian setting was explicitly highlighted as open in the aforementioned works, even in dimension 1. Finally we observe that for reverse KL divergence, a folklore trick yields monotonicity for very general exponential families. All results in this paper were derived by variants of GPT-5.2 Pro. Humans did not provide any proof strategies or intermediate arguments, but only prompted the model to continue developing additional results, and verified and transcribed its proofs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Inference for the proportional odds cumulative logit model with monotonicity constraints for ordinal predictors and ordinal response
The proportional odds cumulative logit model (POCLM) is a standard regression model for an ordinal response. Ordinality of predictors can be incorporated by monotonicity constraints for the corresponding parameters. It i…
Maximum Approximated Likelihood Estimation
Empirical economic research frequently applies maximum likelihood estimation in cases where the likelihood function is analytically intractable. Most of the theoretical literature focuses on maximum simulated likelihood …
A new adjusted maximum likelihood method for the Fay–Herriot small area model
In the context of the Fay–Herriot model, a mixed regression model routinely used to combine information from various sources in small area estimation, certain adjustments to a standard likelihood (e.g., profile, residu…
regressionCounting Defiers in Health Care: A Design-Based Model of an Experiment Can Reveal Evidence Against Monotonicity
We show that a design-based model of an experiment with a binary intervention and outcome can reveal empirical evidence against a ``monotonicity'' assumption that the intervention affects all subjects in weakly the same …
A Scheme for Molecular Computation of Maximum Likelihood Estimators for Log-Linear Models
We propose a novel molecular computing scheme for statistical inference. We focus on the much-studied statistical inference problem of computing maximum likelihood estimators for log-linear models. Our scheme takes log-l…