Regularization-Induced Bias and Consistency in Recursive Least Squares
Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-response (FIR) system identification and, finally, to infinite-impulse-response (IIR) system identification. For each case, we relate the condition number of the regressor matrix to the transient response and rate of convergence of the parameter estimates.
Code (0)
등록된 구현이 없습니다.
Tasks
parameter estimationregressionSimilar Papers 제목 키워드 기반
Coordinate-Independent Robot Model Identification
Robot model identification is commonly performed by least-squares regression on inverse dynamics, but existing formulations measure residuals directly in coordinate force space and therefore depend on the chosen coordina…
Recursive Least Squares with Fading Regularization for Finite-Time Convergence without Persistent Excitation
This paper extends recursive least squares (RLS) to include time-varying regularization. This extension provides flexibility for updating the least squares regularization term in real time. Existing results with constant…
parameter estimationExposure Bias as Epistemic Underidentification in Recursive Forecasting
Recursive multi-step forecasting is usually framed as distribution shift: models are trained on observed histories but deployed on their own predictions. We show this framing is incomplete by proving that, under partial …
Convergence Conditions of Online Regularized Statistical Learning in Reproducing Kernel Hilbert Space With Non-Stationary Data
We study the convergence of recursive regularized learning algorithms in the reproducing kernel Hilbert space (RKHS) with dependent and non-stationary online data streams. Firstly, we study the mean square asymptotic sta…
Implicit Regularization of Accelerated Methods in Hilbert Spaces
We study learning properties of accelerated gradient descent methods for linear least-squares in Hilbert spaces. We analyze the implicit regularization properties of Nesterov acceleration and a variant of heavy-ball in t…