Optimism in Reinforcement Learning with Generalized Linear Function Approximation

2019-12-09 · ICLR 2021 1 · Yining Wang, Ruosong Wang, Simon S. Du, Akshay Krishnamurthy

We design a new provably efficient algorithm for episodic reinforcement learning with generalized linear function approximation. We analyze the algorithm under a new expressivity assumption that we call "optimistic closure," which is strictly weaker than assumptions from prior analyses for the linear setting. With optimistic closure, we prove that our algorithm enjoys a regret bound of $\tilde{O}(\sqrt{d^3 T})$ where $d$ is the dimensionality of the state-action features and $T$ is the number of episodes. This is the first statistically and computationally efficient algorithm for reinforcement learning with generalized linear functions.

📄 PDF Abstract BibTeX arXiv:1912.04136

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Exploration in Model-based Reinforcement Learning with Randomized Reward

2023-01-09 · Lingxiao Wang, Ping Li

Model-based Reinforcement Learning (MBRL) has been widely adapted due to its sample efficiency. However, existing worst-case regret analysis typically requires optimistic planning, which is not realistic in general. In c…

Efficient Exploration Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Entropic Risk-Sensitive Reinforcement Learning: A Meta Regret Framework with Function Approximation

2021-01-01 · Yingjie Fei, Zhuoran Yang, Zhaoran Wang

We study risk-sensitive reinforcement learning with the entropic risk measure and function approximation. We consider the finite-horizon episodic MDP setting, and propose a meta algorithm based on value iteration. We the…

Efficient Exploration reinforcement-learning Reinforcement Learning (RL)

Linear Bellman Completeness Suffices for Efficient Online Reinforcement Learning with Few Actions

2024-06-17 · Noah Golowich, Ankur Moitra

One of the most natural approaches to reinforcement learning (RL) with function approximation is value iteration, which inductively generates approximations to the optimal value function by solving a sequence of regressi…

regression Reinforcement Learning (RL)

A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning

2021-04-28 · Andrew Patterson, Adam White, Martha White

Many reinforcement learning algorithms rely on value estimation, however, the most widely used algorithms -- namely temporal difference algorithms -- can diverge under both off-policy sampling and nonlinear function appr…

reinforcement-learning Reinforcement Learning (RL)

Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature

2021-02-08 · NeurIPS 2021 12 · Kefan Dong, Jiaqi Yang, Tengyu Ma

This paper studies model-based bandit and reinforcement learning (RL) with nonlinear function approximations. We propose to study convergence to approximate local maxima because we show that global convergence is statist…