Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learning
Utilizing messages from teammates can improve coordination in cooperative Multi-agent Reinforcement Learning (MARL). Previous works typically combine raw messages of teammates with local information as inputs for policy. However, neglecting message aggregation poses significant inefficiency for policy learning. Motivated by recent advances in representation learning, we argue that efficient message aggregation is essential for good coordination in cooperative MARL. In this paper, we propose Multi-Agent communication via Self-supervised Information Aggregation (MASIA), where agents can aggregate the received messages into compact representations with high relevance to augment the local policy. Specifically, we design a permutation invariant message encoder to generate common information-aggregated representation from messages and optimize it via reconstructing and shooting future information in a self-supervised manner. Hence, each agent would utilize the most relevant parts of the aggregated representation for decision-making by a novel message extraction mechanism. Furthermore, considering the potential of offline learning for real-world applications, we build offline benchmarks for multi-agent communication, which is the first as we know. Empirical results demonstrate the superiority of our method in both online and offline settings. We also release the built offline benchmarks in this paper as a testbed for communication ability validation to facilitate further future research.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingMulti-agent Reinforcement LearningRepresentation LearningSimilar Papers 제목 키워드 기반
SSoC: Learning Spontaneous and Self-Organizing Communication for Multi-Agent Collaboration
Multi-agent collaboration is required by numerous real-world problems. Although distributed setting is usually adopted by practical systems, local range communication and information aggregation still matter in fulfillin…
Multi-agent Reinforcement LearningReinforcement LearningInformation-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries
On-device federated learning (FL) enables privacy-preserving and personalized model training on resource-constrained devices such as smartphones and IoT nodes. To reduce communication cost, sign-based methods (e.g., sign…
Federated LearningFeature Correlation-guided Knowledge Transfer for Federated Self-supervised Learning
To eliminate the requirement of fully-labeled data for supervised model training in traditional Federated Learning (FL), extensive attention has been paid to the application of Self-supervised Learning (SSL) approaches o…
Feature CorrelationFederated LearningKnowledge DistillationPrivacy Preserving+2Federated Learning with Hypergradient-based Online Update of Aggregation Weights
Federated learning using mobile and Internet of Things devices requires not only the ability to handle heterogeneity of clients' data distributions but also high adaptability to varying communication environments. We pro…
Federated LearningWhat Makes for Hierarchical Vision Transformer?
Recent studies indicate that hierarchical Vision Transformer with a macro architecture of interleaved non-overlapped window-based self-attention \& shifted-window operation is able to achieve state-of-the-art performance…
Instance Segmentationobject-detectionObject DetectionSemantic Segmentation+1