Cooperative Actor-Critic via TD Error Aggregation
In decentralized cooperative multi-agent reinforcement learning, agents can aggregate information from one another to learn policies that maximize a team-average objective function. Despite the willingness to cooperate with others, the individual agents may find direct sharing of information about their local state, reward, and value function undesirable due to privacy issues. In this work, we introduce a decentralized actor-critic algorithm with TD error aggregation that does not violate privacy issues and assumes that communication channels are subject to time delays and packet dropouts. The cost we pay for making such weak assumptions is an increased communication burden for every agent as measured by the dimension of the transmitted data. Interestingly, the communication burden is only quadratic in the graph size, which renders the algorithm applicable in large networks. We provide a convergence analysis under diminishing step size to verify that the agents maximize the team-average objective function.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-agent Reinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations
Multi-agent reinforcement learning (MARL) has been increasingly explored to learn the cooperative policy towards maximizing a certain global reward. Many existing studies take advantage of graph neural networks (GNN) in …
DecoderGraph AttentionMulti-agent Reinforcement LearningRelay-Assisted Cooperative Federated Learning
Federated learning (FL) has recently emerged as a promising technology to enable artificial intelligence (AI) at the network edge, where distributed mobile devices collaboratively train a shared AI model under the coordi…
Federated LearningDual-Interaction-Aware Cooperative Control Strategy for Alleviating Mixed Traffic Congestion
As Intelligent Transportation System (ITS) develops, Connected and Automated Vehicles (CAVs) are expected to significantly reduce traffic congestion through cooperative strategies, such as in bottleneck areas. However, t…
Multi-agent Reinforcement LearningDeveloping cooperative policies for multi-stage tasks
This paper proposes the Cooperative Soft Actor Critic (CSAC) method of enabling consecutive reinforcement learning agents to cooperatively solve a long time horizon multi-stage task. This method is achieved by modifying …
Reinforcement Learning (RL)Strategic learning for disturbance rejection in multi-agent systems: Nash and Minmax in graphical games
This article investigates the optimal control problem with disturbance rejection for discrete-time multi-agent systems under cooperative and non-cooperative graphical games frameworks. Given the practical challenges of o…