Maximum likelihood smoothing estimation in state-space models: An incomplete-information based approach
This paper revisits classical works of Rauch (1963, et al. 1965) and develops a novel method for maximum likelihood (ML) smoothing estimation from incomplete information/data of stochastic state-space systems. Score function and conditional observed information matrices of incomplete data are introduced and their distributional identities are established. Using these identities, the ML smoother $\widehat{x}_{k\vert n}^s =\argmax_{x_k} \log f(x_k,\widehat{x}_{k+1\vert n}^s, y_{0:n}\vert\theta)$, $k\leq n-1$, is presented. The result shows that the ML smoother gives an estimate of state $x_k$ with more adherence of loglikehood having less standard errors than that of the ML state estimator $\widehat{x}_k=\argmax_{x_k} \log f(x_k,y_{0:k}\vert\theta)$, with $\widehat{x}_{n\vert n}^s=\widehat{x}_n$. Recursive estimation is given in terms of an EM-gradient-particle algorithm which extends the work of \cite{Lange} for ML smoothing estimation. The algorithm has an explicit iteration update which lacks in (\cite{Ramadan}) EM-algorithm for smoothing. A sequential Monte Carlo method is developed for valuation of the score function and observed information matrices. A recursive equation for the covariance matrix of estimation error is developed to calculate the standard errors. In the case of linear systems, the method shows that the Rauch-Tung-Striebel (RTS) smoother is a fully efficient smoothing state-estimator whose covariance matrix coincides with the Cram\'er-Rao lower bound, the inverse of expected information matrix. Furthermore, the RTS smoother coincides with the Kalman filter having less covariance matrix. Numerical studies are performed, confirming the accuracy of the main results.
Code (0)
등록된 구현이 없습니다.
Tasks
State Space ModelsSimilar Papers 제목 키워드 기반
Newton-based maximum likelihood estimation in nonlinear state space models
Maximum likelihood (ML) estimation using Newton's method in nonlinear state space models (SSMs) is a challenging problem due to the analytical intractability of the log-likelihood and its gradient and Hessian. We estimat…
parameter estimationState Space ModelsvalidBoosting in Univariate Nonparametric Maximum Likelihood Estimation
Nonparametric maximum likelihood estimation is intended to infer the unknown density distribution while making as few assumptions as possible. To alleviate the over parameterization in nonparametric data fitting, smoothi…
Token-level and sequence-level loss smoothing for RNN language models
Despite the effectiveness of recurrent neural network language models, their maximum likelihood estimation suffers from two limitations. It treats all sentences that do not match the ground truth as equally poor, ignorin…
Image CaptioningMachine TranslationTranslationFinite-Sample Maximum Likelihood Estimation of Location
We consider 1-dimensional location estimation, where we estimate a parameter $\lambda$ from $n$ samples $\lambda + \eta_i$, with each $\eta_i$ drawn i.i.d. from a known distribution $f$. For fixed $f$ the maximum-likelih…
Asymptotic Properties of Recursive Maximum Likelihood Estimation in Non-Linear State-Space Models
Using stochastic gradient search and the optimal filter derivative, it is possible to perform recursive (i.e., online) maximum likelihood estimation in a non-linear state-space model. As the optimal filter and its deriva…
State Space Models