홈 › Papers

Align-RUDDER: Learning From Few Demonstrations by Reward Redistribution

2020-09-29 · Vihang P. Patil, Markus Hofmarcher, Marius-Constantin Dinu, Matthias Dorfer, Patrick M. Blies, Johannes Brandstetter, Jose A. Arjona-Medina, Sepp Hochreiter

Reinforcement learning algorithms require many samples when solving complex hierarchical tasks with sparse and delayed rewards. For such complex tasks, the recently proposed RUDDER uses reward redistribution to leverage steps in the Q-function that are associated with accomplishing sub-tasks. However, often only few episodes with high rewards are available as demonstrations since current exploration strategies cannot discover them in reasonable time. In this work, we introduce Align-RUDDER, which utilizes a profile model for reward redistribution that is obtained from multiple sequence alignment of demonstrations. Consequently, Align-RUDDER employs reward redistribution effectively and, thereby, drastically improves learning on few demonstrations. Align-RUDDER outperforms competitors on complex artificial tasks with delayed rewards and few demonstrations. On the Minecraft ObtainDiamond task, Align-RUDDER is able to mine a diamond, though not frequently. Code is available at https://github.com/ml-jku/align-rudder. YouTube: https://youtu.be/HO-_8ZUl-UY

📄 PDF Abstract BibTeX arXiv:2009.14108

Code (1)

ml-jku/align-rudder 공식 구현

Tasks

General Reinforcement LearningMinecraftMultiple Sequence AlignmentSafe Exploration

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

RUDDER: Return Decomposition for Delayed Rewards

2018-06-20 · NeurIPS 2019 12 · Jose A. Arjona-Medina, Michael Gillhofer, Michael Widrich, Thomas Unterthiner 외

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards.…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach

2023-05-28 · NeurIPS 2023 11 · Yudi Zhang, Yali Du, Biwei Huang, Ziyan Wang 외

A major challenge in reinforcement learning is to determine which state-action pairs are responsible for future rewards that are delayed. Reward redistribution serves as a solution to re-assign credits for each time step…

reinforcement-learning Reinforcement Learning

Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning

2024-12-15 · Yun Qu, Yuhang Jiang, Boyuan Wang, Yixiu Mao 외

Reinforcement learning (RL) often encounters delayed and sparse feedback in real-world applications, even with only episodic rewards. Previous approaches have made some progress in reward redistribution for credit assign…

Decision Making Large Language Model reinforcement-learning Reinforcement Learning+1

Likelihood Reward Redistribution

2025-03-20 · Minheng Xiao, Zhenbang Jiao

In many practical reinforcement learning scenarios, feedback is provided only at the end of a long horizon, leading to sparse and delayed rewards. Existing reward redistribution methods typically assume that per-step rew…

MuJoCo

RREDCoT: Segment-Level Reward Redistribution for Reasoning Models

2026-06-04 · Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger, Sepp Hochreiter arxiv

Recent advancements in reasoning language models have been driven by Reinforcement Learning (RL) fine-tuning. Most often, these rely on the Group Relative Policy Optimization (GRPO) algorithm or modifications thereof to …

Reinforcement Learning