An Approximate MSE Expression for Maximum Likelihood and Other Implicitly Defined Estimators of Non-Random Parameters (extended version)
An approximate mean square error (MSE) expression for the performance analysis of implicitly defined estimators of non-random parameters is proposed. An implicitly defined estimator (IDE) declares the minimizer/maximizer of a selected cost/reward function as the parameter estimate. The maximum likelihood (ML) and the least squares estimators are among the well known examples of this class. In this paper, an exact MSE expression for implicitly defined estimators with a symmetric and unimodal objective function is given. It is shown that the expression reduces to the Cramer-Rao lower bound (CRLB) and misspecified CRLB in the large sample size regime for ML and misspecified ML estimation, respectively. The expression is shown to yield the Ziv-Zakai bound (without the valley filling function) for the maximum a posteriori (MAP) estimator when it is used in a Bayesian setting, that is, when an a-priori distribution is assigned to the unknown parameter. In addition, extension of the suggested expression to the case of nuisance parameters is studied and some approximations are given to ease the computations for this case. Numerical results indicate that the suggested MSE expression not only predicts the estimator performance in the asymptotic region; but it is also applicable for the threshold region analysis, even for IDEs whose objective functions do not satisfy the symmetry and unimodality assumptions. Advantages of the suggested MSE expression are its conceptual simplicity and its relatively straightforward numerical calculation due to the reduction of the estimation problem to a binary hypothesis testing problem, similar to the usage of Ziv-Zakai bounds in random parameter estimation problems.
Code (0)
등록된 구현이 없습니다.
Tasks
parameter estimationSimilar Papers 제목 키워드 기반
Restricted maximum-likelihood method for learning latent variance components in gene expression data with known and unknown confounders
Random effect models are popular statistical models for detecting and correcting spurious sample correlations due to hidden confounders in genome-wide gene expression data. In applications where some confounding factors …
Neural Likelihood Surfaces for Spatial Processes with Computationally Intensive or Intractable Likelihoods
In spatial statistics, fast and accurate parameter estimation, coupled with a reliable means of uncertainty quantification, can be challenging when fitting a spatial process to real-world data because the likelihood func…
parameter estimationUncertainty QuantificationMaximum Proxy-Likelihood Estimation for Non-autoregressive Machine Translation
Maximum Likelihood Estimation (MLE) is commonly used in machine translation, where models with higher likelihood are assumed to perform better in translation. However, this assumption does not hold in the non-autoregress…
Machine TranslationTranslationMaximum Approximated Likelihood Estimation
Empirical economic research frequently applies maximum likelihood estimation in cases where the likelihood function is analytically intractable. Most of the theoretical literature focuses on maximum simulated likelihood …
Maximum Likelihood Reinforcement Learning
Reinforcement learning (RL) is the method of choice for training models in setups where the objective function can only be evaluated by sampling from the model. Our key observation is that when the feedback is terminal a…
Reinforcement LearningCode Generation