Distributed Soft Actor-Critic with Multivariate Reward Representation and Knowledge Distillation

2019-11-29 · Dmitry Akimov

In this paper, we describe NeurIPS 2019 Learning to Move - Walk Around challenge physics-based environment and present our solution to this competition which scored 1303.727 mean reward points and took 3rd place. Our method combines recent advances from both continuous- and discrete-action space reinforcement learning, such as Soft Actor-Critic and Recurrent Experience Replay in Distributed Reinforcement Learning. We trained our agent in two stages: to move somewhere at the first stage and to follow the target velocity field at the second stage. We also introduce novel Q-function split technique, which we believe facilitates the task of training an agent, allows critic pretraining and reusing it for solving harder problems, and mitigate reward shaping design efforts.

📄 PDF Abstract BibTeX arXiv:1911.13056

Code (1)

DEAkimov/learn_to_move 공식 구현 pytorch

Tasks

Knowledge Distillation reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Discriminator Soft Actor Critic without Extrinsic Rewards

2020-01-19 · Daichi Nishio, Daiki Kuyoshi, Toi Tsuneda, Satoshi Yamane

It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer f…

Imitation Learning Q-Learning reinforcement-learning Reinforcement Learning+1

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning

2024-08-04

In this paper, we propose an off-policy deep reinforcement learning (DRL) method utilizing the average reward criterion. While most existing DRL methods employ the discounted reward criterion, this can potentially lead t…

Loss- and Reward-Weighting for Efficient Distributed Reinforcement Learning

2023-04-25 · Martin Holen, Per-Arne Andersen, Kristian Muri Knausgård, Morten Goodwin

This paper introduces two learning schemes for distributed agents in Reinforcement Learning (RL) environments, namely Reward-Weighted (R-Weighted) and Loss-Weighted (L-Weighted) gradient merger. The R/L weighted methods …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Soft Actor-Critic for Sparse-Reward Long-Horizon Reinforcement Learning

2026-07-26 · Zahra Abdalla Elashaal, Afef Hfaiedh, Nahla Khraief, Issmail Ellabib 외 arxiv

Exploration in sparse-reward long-horizon tasks poses significant challenges for reinforcement learning. To address these challenges, we propose a two-level Hierarchical Reinforcement Learning (HRL) framework. The first …

Hierarchical Reinforcement Learning Continuous Control

Average Reward Reinforcement Learning for Wireless Radio Resource Management

2025-01-12 · Kun Yang, Jing Yang, Cong Shen

In this paper, we address a crucial but often overlooked issue in applying reinforcement learning (RL) to radio resource management (RRM) in wireless communications: the mismatch between the discounted reward RL formulat…

Management reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)