Behavior Constraining in Weight Space for Offline Reinforcement Learning

2021-07-12 · Phillip Swazinna, Steffen Udluft, Daniel Hein, Thomas Runkler

In offline reinforcement learning, a policy needs to be learned from a single pre-collected dataset. Typically, policies are thus regularized during training to behave similarly to the data generating policy, by adding a penalty based on a divergence between action distributions of generating and trained policy. We propose a new algorithm, which constrains the policy directly in its weight space instead, and demonstrate its effectiveness in experiments.

📄 PDF Abstract BibTeX arXiv:2107.05479

Code (1)

siemens/industrialbenchmark 공식 구현 tf

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Minimalist Approach to Offline Reinforcement Learning

2021-06-12 · NeurIPS 2021 12 · Scott Fujimoto, Shixiang Shane Gu

Offline reinforcement learning (RL) defines the task of learning from a fixed batch of data. Due to errors in value estimation from out-of-distribution actions, most offline RL algorithms take the approach of constrainin…

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

PLAS: Latent Action Space for Offline Reinforcement Learning

2020-11-14 · Wenxuan Zhou, Sujay Bajracharya, David Held

The goal of offline reinforcement learning is to learn a policy from a fixed dataset, without further interactions with the environment. This setting will be an increasingly more important paradigm for real-world applica…

continuous-control Continuous Control Deformable Object Manipulation reinforcement-learning+2

Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model

2024-01-19 · Yinan Zheng, Jianxiong Li, Dongjie Yu, Yujie Yang 외

Safe offline RL is a promising way to bypass risky online interactions towards safe policy learning. Most existing methods only enforce soft constraints, i.e., constraining safety violations in expectation below threshol…

Offline RL reinforcement-learning Reinforcement Learning

Matrix Estimation for Offline Reinforcement Learning with Low-Rank Structure

2023-05-24 · Xumei Xi, Christina Lee Yu, Yudong Chen

We consider offline Reinforcement Learning (RL), where the agent does not interact with the environment and must rely on offline data collected using a behavior policy. Previous works provide policy evaluation guarantees…

Matrix Completion reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

ReDS: Offline RL With Heteroskedastic Datasets via Support Constraints

2023-09-21 · NeurIPS 2023 11

Offline reinforcement learning (RL) learns policies entirely from static datasets. Practical applications of offline RL will inevitably require learning from datasets where the variability of demonstrated behaviors chang…