Scale-free adaptive planning for deterministic dynamics & discounted rewards
We address the problem of planning in an environment with deterministic dynamics and stochastic rewards with discounted returns. The optimal value function is not known, nor are the rewards bounded. We propose Platypoos, a simple scale-free planning algorithm that adapts to the unknown scale and smoothness of the reward function. We provide a sample complexity analysis for Platypoos that improves upon prior work and holds simultaneously over a broad range of discount factors and reward scales, without the algorithm knowing them. We also establish a matching lower bound showing our analysis is optimal up to constants.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Scalable Solution Methods for Dec-POMDPs with Deterministic Dynamics
Many high-level multi-agent planning problems, including multi-robot navigation and path planning, can be effectively modeled using deterministic actions and observations. In this work, we focus on such domains and intro…
Robot NavigationLearning Latent Dynamics for Planning from Pixels
Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning…
continuous-controlContinuous ControlMotion PlanningVariational InferenceAdaptive Online Planning for Continual Lifelong Learning
We study learning control in an online reset-free lifelong learning scenario, where mistakes can compound catastrophically into the future and the underlying dynamics of the environment may change. Traditional model-free…
Lifelong learningReinforcement LearningIn-Context Planning with Latent Temporal Abstractions
Planning-based reinforcement learning for continuous control is bottlenecked by two practical issues: planning at primitive time scales leads to prohibitive branching and long horizons, while real environments are freque…
Reinforcement LearningContinuous ControlOffline RLModel-Driven Policy Optimization in Differentiable Simulators via Stochastic Exploration
Differentiable planning enables gradient-based optimization of decision-making problems by leveraging differentiable models of system dynamics. However, in highly nonlinear and hybrid discrete-continuous domains, the res…