Meta Attention For Off-Policy Actor-Critic

2021-09-29 · Jiateng Huang, Wanrong Huang, Long Lan, Dan Wu

Off-Policy Actor-Critic methods can effectively exploit past experiences and thus they have achieved great success in various reinforcement learning tasks. In many image-based and multi-source tasks, attention mechanism has been employed in Actor-Critic methods to improve their sampling efﬁciency. In this paper, we propose a meta attention method for state-based reinforcement learning tasks, which combines attention mechanism and meta-learning based on the Off-Policy Actor-Critic framework. Unlike previous attention-based work, our meta attention method introduces attention in the actor and the critic of the typical Actor-Critic framework rather than in multiple pixels of an image or multiple information sources. In contrast to existing meta-learning methods, the proposed meta-attention approach is able to function in both the gradient-based training phase and the agent's decision-making process. The experimental results demonstrate the superiority of our meta-attention method in various continuous control tasks, which are based on the Off-Policy Actor-Critic methods including DDPG, TD3, and SAC.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Decision Making Meta-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Weight Decay 설명 없음

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Batch Normalization 설명 없음

Average Pooling 설명 없음

Target Policy Smoothing Target Policy Smoothing is a regularization strategy for the value function in reinforcement learning. Deterministic policies can overfit to narrow peaks in the value…

Dilated Convolution 설명 없음

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Online Meta-Critic Learning for Off-Policy Actor-Critic Methods

2020-03-11 · NeurIPS 2020 12 · Wei Zhou, Yiying Li, Yongxin Yang, Huaimin Wang 외

Off-Policy Actor-Critic (Off-PAC) methods have proven successful in a variety of continuous control tasks. Normally, the critic's action-value function is updated using temporal-difference, and the critic in turn provide…

continuous-control Continuous Control Meta-Learning Reinforcement Learning

Interpretable Meta-Reinforcement Learning with Actor-Critic Method

2021-01-01 · Xingyuan Liang, Xu-Ying Liu

Meta-reinforcement learning (meta-RL) algorithms have successfully trained agent systems to perform well on different tasks within only few updates. However, in gradient-based meta-RL algorithms, the Q-function at adapta…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

EVOM: Agentic Meta-Evolution of Actor-Critic Architectures for Reinforcement Learning

2026-06-24 · Boyun Zhang, Chao Wang, Kai Wu arxiv

In actor-critic reinforcement learning, network architectures are typically manually designed. Automating this design is challenging because each candidate must be trained before evaluation, and the design space is open-…

Reinforcement Learning

Learning to Explore with Meta-Policy Gradient

2018-03-13 · Tianbing Xu, Qiang Liu, Liang Zhao, Jian Peng

The performance of off-policy learning, including deep Q-learning and deep deterministic policy gradient (DDPG), critically depends on the choice of the exploration policy. Existing exploration methods are mostly based o…

Q-Learning Reinforcement Learning Reinforcement Learning (RL)

Learning to Explore via Meta-Policy Gradient

2018-07-01 · ICML 2018 7 · Tianbing Xu, Qiang Liu, Liang Zhao, Jian Peng

continuous-control Continuous Control Q-Learning reinforcement-learning+2