Contrastive learning-based agent modeling for deep reinforcement learning

2023-12-30 · Wenhao Ma, Yu-Cheng Chang, Jie Yang, Yu-Kai Wang, Chin-Teng Lin

Multi-agent systems often require agents to collaborate with or compete against other agents with diverse goals, behaviors, or strategies. Agent modeling is essential when designing adaptive policies for intelligent machine agents in multiagent systems, as this is the means by which the ego agent understands other agents' behavior and extracts their meaningful policy representations. These representations can be used to enhance the ego agent's adaptive policy which is trained by reinforcement learning. However, existing agent modeling approaches typically assume the availability of local observations from other agents (modeled agents) during training or a long observation trajectory for policy adaption. To remove these constrictive assumptions and improve agent modeling performance, we devised a Contrastive Learning-based Agent Modeling (CLAM) method that relies only on the local observations from the ego agent during training and execution. With these observations, CLAM is capable of generating consistent high-quality policy representations in real-time right from the beginning of each episode. We evaluated the efficacy of our approach in both cooperative and competitive multi-agent environments. Our experiments demonstrate that our approach achieves state-of-the-art on both cooperative and competitive tasks, highlighting the potential of contrastive learning-based agent modeling for enhancing reinforcement learning.

📄 PDF Abstract BibTeX arXiv:2401.00132

Code (0)

등록된 구현이 없습니다.

Tasks

Contrastive LearningDeep Reinforcement Learningreinforcement-learning Reinforcement Learning

CoBERL: Contrastive BERT for Reinforcement Learning

2021-07-12 · ICML Workshop URL 2021 7 · Andrea Banino, Adrià Puidomenech Badia, Jacob Walker, Tim Scholtes 외

Many reinforcement learning (RL) agents require a large amount of experience to solve tasks. We propose Contrastive BERT for RL (CoBERL), an agent that combines a new contrastive loss and a hybrid LSTM-transformer archit…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Task-Relevant Representations with Selective Contrast for Reinforcement Learning in a Real-World Application

2021-06-13 · ICML Workshop URL 2021 7 · Flemming Brieger, Daniel Alexander Braun, Sascha Lange

We use contrastive learning to obtain task-relevant state-representations from images for reinforcement learning in a real-world system. To test the quality of the representations, an agent is trained with reinforcement …

Contrastive Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

R-LAtte: Attention Module for Visual Control via Reinforcement Learning

2021-01-01 · Mandi Zhao, Qiyang Li, Aravind Srinivas, Ignasi Clavera 외

Attention mechanisms are generic inductive biases that have played a critical role in improving the state-of-the-art in supervised learning, unsupervised pre-training and generative modeling for multiple domains includin…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Unsupervised Pre-training

Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisation

2026-05-13 · Asim Osman, Sasha Abramowitz, Mark Bergh, Ulrich Armel Mbou Sob 외 arxiv

Contrastive reinforcement learning (CRL) learns goal-conditioned Q-values through a contrastive objective over state-action and goal representations, removing the need for hand-crafted reward functions. Despite impressiv…

Self-Supervised Learning Reinforcement Learning

CURL: Contrastive Unsupervised Representation Learning for Reinforcement Learning

2020-01-01 · ICML 2020 1 · Michael Laskin, Pieter Abbeel, Aravind Srinivas

Reinforcement Learning for control tasks where the agent learns from raw high dimensional pixels has proven to be difficult and sample-inefficient. Operating on high-dimensional observational input poses a challenging c…