Statistical Inference for Online Algorithms

2025-05-22 · Selina Carter, Arun K Kuchibhotla

Construction of confidence intervals and hypothesis tests for functionals based on asymptotically normal estimators is a classical topic in statistical inference. The simplest and in many cases optimal inference procedure is the Wald interval or the likelihood ratio test, both of which require an estimator and an estimate of the asymptotic variance of the estimator. Estimators obtained from online/sequential algorithms forces one to consider the computational aspects of the inference problem, i.e., one cannot access all of the data as many times as needed. Several works on this topic explored the online estimation of asymptotic variance. In this article, we propose computationally efficient, rate-optimal, and asymptotically valid confidence regions based on the output of online algorithms {\em without} estimating the asymptotic variance. As a special case, this implies inference from any algorithm that yields an asymptotically normal estimator. We focus our efforts on stochastic gradient descent with Polyak averaging to understand the practical performance of the proposed method.

📄 PDF Abstract BibTeX arXiv:2505.17300

Code (1)

Arun-Kuchibhotla/HulC 공식 구현

Tasks

valid

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Polyak Averaging Polyak Averaging is an optimization technique that sets final parameters to an average of (recent) parameters visited in the optimization trajectory. Specifically if in $t$…

Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning

2021-08-08 · Pratik Ramprasad, Yuantong Li, Zhuoran Yang, Zhaoran Wang 외

The recent emergence of reinforcement learning has created a demand for robust statistical inference methods for the parameter estimates computed using these algorithms. Existing methods for statistical inference in onli…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Online Statistical Inference in Decision-Making with Matrix Context

2022-12-21 · Qiyu Han, Will Wei Sun, Yichen Zhang

The study of online decision-making problems that leverage contextual information has drawn notable attention due to their significant applications in fields ranging from healthcare to autonomous systems. In modern appli…

Decision Making Sequential Decision Making

Statistical Inference After Adaptive Sampling for Longitudinal Data

2022-02-14 · Kelly W. Zhang, Lucas Janson, Susan A. Murphy

Online reinforcement learning and other adaptive sampling algorithms are increasingly used in digital intervention experiments to optimize treatment delivery for users over time. In this work, we focus on longitudinal us…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Online Statistical Inference of Constant Sample-averaged Q-Learning

2026-03-27 · Saunak Kumar Panda, Tong Li, Ruiqi Liu, Yisha Xiang arxiv

Reinforcement learning algorithms have been widely used for decision-making tasks in various domains. However, the performance of these algorithms can be impacted by high variance and instability, particularly in environ…

Reinforcement Learning

Recursive Entropic Variational Inference for Nonlinear State-Space Models

2025-11-19 · Hany Abdulsamad, Ángel F. García-Fernández, Simo Särkkä arxiv

We present a class of algorithms for state estimation in nonlinear, non-Gaussian state-space models. Our approach is based on a variational Lagrangian formulation that casts Bayesian inference as a sequence of entropic t…

Bayesian Inference