Sample-efficient and Scalable Exploration in Continuous-Time RL

2025-10-28 · Klemens Iten, Lenart Treven, Bhavya Sukhija, Florian Dörfler, Andreas Krause arxiv

Reinforcement learning algorithms are typically designed for discrete-time dynamics, even though the underlying real-world control systems are often continuous in time. In this paper, we study the problem of continuous-time reinforcement learning, where the unknown system dynamics are represented using nonlinear ordinary differential equations (ODEs). We leverage probabilistic models, such as Gaussian processes and Bayesian neural networks, to learn an uncertainty-aware model of the underlying ODE. Our algorithm, COMBRL, greedily maximizes a weighted sum of the extrinsic reward and model epistemic uncertainty. This yields a scalable and sample-efficient approach to continuous-time model-based RL. We show that COMBRL achieves sublinear regret in the reward-driven setting, and in the unsupervised RL setting (i.e., without extrinsic rewards), we provide a sample complexity bound. In our experiments, we evaluate COMBRL in both standard and unsupervised RL settings and demonstrate that it scales better, is more sample-efficient than prior methods, and outperforms baselines across several deep RL tasks.

📄 PDF Abstract BibTeX arXiv:2510.24482

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Gaussian Processes

Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow

2026-01-27 · Yunyue Wei, Chenhui Zuo, Yanan Sui arxiv

Controlling high-dimensional systems in biological and robotic applications is challenging due to expansive state-action spaces, where effective exploration is critical. Commonly used exploration strategies in reinforcem…

Dimensionality Reduction Reinforcement Learning Continuous Control

Deep Coherent Exploration For Continuous Control

2021-01-01 · Yijie Zhang, Herke van Hoof

In policy search methods for reinforcement learning (RL), exploration is often performed by injecting noise either in action space at each step independently or in parameter space over each full trajectory. In prior work…

continuous-control Continuous Control Reinforcement Learning (RL)

Information Content Exploration

2023-10-10 · Jacob Chmura, Hasham Burhani, Xiao Qi Shi

Sparse reward environments are known to be challenging for reinforcement learning agents. In such environments, efficient and scalable exploration is crucial. Exploration is a means by which an agent gains information ab…

Efficient Exploration reinforcement-learning Reinforcement Learning

SVDE: Scalable Value-Decomposition Exploration for Cooperative Multi-Agent Reinforcement Learning

2023-03-16 · Shuhan Qi, Shuhao Zhang, Qiang Wang, Jiajia Zhang 외

Value-decomposition methods, which reduce the difficulty of a multi-agent system by decomposing the joint state-action space into local observation-action spaces, have become popular in cooperative multi-agent reinforcem…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Starcraft+1

Data-dependent Exploration for Online Reinforcement Learning from Human Feedback

2026-05-06 · Zhen-Yu Zhang, Yuting Tang, Jiandong Zhang, Lanjihong Ma 외 arxiv

Online reinforcement learning from human feedback (RLHF) has emerged as a promising paradigm for aligning large language models (LLMs) by continuously collecting new preference feedback during training. A foundational ch…

Reinforcement Learning