Deterministic Sequencing of Exploration and Exploitation for Reinforcement Learning

2022-09-12 · Piyush Gupta, Vaibhav Srivastava

We propose Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm with interleaving exploration and exploitation epochs for model-based RL problems that aim to simultaneously learn the system model, i.e., a Markov decision process (MDP), and the associated optimal policy. During exploration, DSEE explores the environment and updates the estimates for expected reward and transition probabilities. During exploitation, the latest estimates of the expected reward and transition probabilities are used to obtain a robust policy with high probability. We design the lengths of the exploration and exploitation epochs such that the cumulative regret grows as a sub-linear function of time.

📄 PDF Abstract BibTeX arXiv:2209.05408

Code (0)

등록된 구현이 없습니다.

Tasks

Efficient Explorationreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Robot Gaussian Process Estimation and Coverage: A Deterministic Sequencing Algorithm and Regret Analysis

2021-01-12 · Lai Wei, Andrew McDonald, Vaibhav Srivastava

We study the problem of distributed multi-robot coverage over an unknown, nonuniform sensory field. Modeling the sensory field as a realization of a Gaussian Process and using Bayesian techniques, we devise a policy whic…

Chaos-based reinforcement learning with TD3

2024-05-15 · Toshitaka Matsuki, Yusuke Sakemi, Kazuyuki Aihara

Chaos-based reinforcement learning (CBRL) is a method in which the agent's internal chaotic dynamics drives exploration. This approach offers a model for considering how the biological brain can create variability in its…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

On Abruptly-Changing and Slowly-Varying Multiarmed Bandit Problems

2018-02-23 · Lai Wei, Vaibhav Srivastava

We study the non-stationary stochastic multiarmed bandit (MAB) problem and propose two generic algorithms, namely, the limited memory deterministic sequencing of exploration and exploitation (LM-DSEE) and the Sliding-Win…

Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward

2025-12-18 · Peter Chen, Xiaopeng Li, Ziniu Li, Wotao Yin 외 arxiv

This paper examines the exploration-exploitation trade-off in reinforcement learning with verifiable rewards (RLVR), a framework for improving the reasoning of Large Language Models (LLMs). Recent studies suggest that RL…

Reinforcement Learning Mathematical Reasoning

Dealing with uncertainty: balancing exploration and exploitation in deep recurrent reinforcement learning

2023-10-12 · Valentina Zangirolami, Matteo Borrotti

Incomplete knowledge of the environment leads an agent to make decisions under uncertainty. One of the major dilemmas in Reinforcement Learning (RL) where an autonomous agent has to balance two contrasting needs in makin…

Autonomous Driving reinforcement-learning Reinforcement Learning (RL)