Replicable Reinforcement Learning with Linear Function Approximation

2025-09-10 · Eric Eaton, Marcel Hussing, Michael Kearns, Aaron Roth, Sikata Bela Sengupta, Jessica Sorrell arxiv

Replication of experimental results has been a challenge faced by many scientific disciplines, including the field of machine learning. Recent work on the theory of machine learning has formalized replicability as the demand that an algorithm produce identical outcomes when executed twice on different samples from the same distribution. Provably replicable algorithms are especially interesting for reinforcement learning (RL), where algorithms are known to be unstable in practice. While replicable algorithms exist for tabular RL settings, extending these guarantees to more practical function approximation settings has remained an open problem. In this work, we make progress by developing replicable methods for linear function approximation in RL. We first introduce two efficient algorithms for replicable random design regression and uncentered covariance estimation, each of independent interest. We then leverage these tools to provide the first provably efficient replicable RL algorithms for linear Markov decision processes in both the generative model and episodic settings. Finally, we evaluate our algorithms experimentally and show how they can inspire more consistent neural policies.

📄 PDF Abstract BibTeX arXiv:2509.08660

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Replicable Clustering

2023-02-20 · NeurIPS 2023 11

We design replicable algorithms in the context of statistical clustering under the recently introduced notion of replicability from Impagliazzo et al. [2022]. According to this definition, a clustering algorithm is repli…

Clustering

Linear Feature Encoding for Reinforcement Learning

2016-12-01 · NeurIPS 2016 12 · Zhao Song, Ronald E. Parr, Xuejun Liao, Lawrence Carin

Feature construction is of vital importance in reinforcement learning, as the quality of a value function or policy is largely determined by the corresponding features. The recent successes of deep reinforcement learning…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Replicable Reinforcement Learning

2023-05-24 · NeurIPS 2023 11

The replicability crisis in the social, behavioral, and data sciences has led to the formulation of algorithm frameworks for replicability -- i.e., a requirement that an algorithm produce identical outputs (with high pro…

reinforcement-learning Reinforcement Learning

Replicable Bandits with UCB based Exploration

2026-04-21 · Rohan Deb, Udaya Ghai, Karan Singh, Arindam Banerjee arxiv

We study replicable algorithms for stochastic multi-armed bandits (MAB) and linear bandits with UCB (Upper Confidence Bound) based exploration. A bandit algorithm is $ρ$-replicable if two executions using shared internal…

Multi-Armed Bandits

Reinforcement Learning with Function Approximation: From Linear to Nonlinear

2023-02-20 · Jihao Long, Jiequn Han

Function approximation has been an indispensable component in modern reinforcement learning algorithms designed to tackle problems with large state spaces in high dimensions. This paper reviews recent results on error an…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)