Rethinking Optimal Transport in Offline Reinforcement Learning

2024-10-17 · Arip Asadulaev, Rostislav Korst, Alexander Korotin, Vage Egiazarian, Andrey Filchenkov, Evgeny Burnaev

We propose a novel algorithm for offline reinforcement learning using optimal transport. Typically, in offline reinforcement learning, the data is provided by various experts and some of them can be sub-optimal. To extract an efficient policy, it is necessary to \emph{stitch} the best behaviors from the dataset. To address this problem, we rethink offline reinforcement learning as an optimal transportation problem. And based on this, we present an algorithm that aims to find a policy that maps states to a \emph{partial} distribution of the best expert actions for each given state. We evaluate the performance of our algorithm on continuous control problems from the D4RL suite and demonstrate improvements over existing methods.

📄 PDF Abstract BibTeX arXiv:2410.14069

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control D4RL reinforcement-learning Reinforcement Learning

Optimal Transport for Offline Imitation Learning

2023-03-24 · Yicheng Luo, Zhengyao Jiang, samuel cohen, Edward Grefenstette 외

With the advent of large datasets, offline reinforcement learning (RL) is a promising framework for learning good decision-making policies without the need to interact with the real environment. However, offline RL requi…

D4RL Decision Making Imitation Learning Offline RL+1

Align Your Intents: Offline Imitation Learning via Optimal Transport

2024-02-20 · Maksim Bobrin, Nazar Buzun, Dmitrii Krylov, Dmitry V. Dylov

Offline Reinforcement Learning (RL) addresses the problem of sequential decision-making by learning optimal policy through pre-collected data, without interacting with the environment. As yet, it has remained somewhat im…

D4RL Decision Making Imitation Learning Offline RL+2

Video-Based Optimal Transport for Feedback-Efficient Offline Preference-Based Reinforcement Learning

2026-06-15 · Tung M. Luu, Hwanhee Kim, Younghwan Lee, Chang D. Yoo arxiv

Conveying complex objectives to reinforcement learning (RL) agents often requires meticulous reward engineering. Preference-based RL (PbRL) offers a promising alternative by learning reward functions from human feedback,…

Reinforcement Learning

Offline Reinforcement Learning with Wasserstein Regularization via Optimal Transport Maps

2025-07-14 · Motoki Omura, Yusuke Mukuta, Kazuki Ota, Takayuki Osa 외 arxiv

Offline reinforcement learning (RL) aims to learn an optimal policy from a static dataset, making it particularly valuable in scenarios where data collection is costly, such as robotics. A major challenge in offline RL i…

Reinforcement Learning Offline RL

Online Multimodal Transportation Planning using Deep Reinforcement Learning

2021-05-18 · Amirreza Farahani, Laura Genga, Remco Dijkman

In this paper we propose a Deep Reinforcement Learning approach to solve a multimodal transportation planning problem, in which containers must be assigned to a truck or to trains that will transport them to their destin…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)