DePAint: A Decentralized Safe Multi-Agent Reinforcement Learning Algorithm considering Peak and Average Constraints
The domain of safe multi-agent reinforcement learning (MARL), despite its potential applications in areas ranging from drone delivery and vehicle automation to the development of zero-energy communities, remains relatively unexplored. The primary challenge involves training agents to learn optimal policies that maximize rewards while adhering to stringent safety constraints, all without the oversight of a central controller. These constraints are critical in a wide array of applications. Moreover, ensuring the privacy of sensitive information in decentralized settings introduces an additional layer of complexity, necessitating innovative solutions that uphold privacy while achieving the system's safety and efficiency goals. In this paper, we address the problem of multi-agent policy optimization in a decentralized setting, where agents communicate with their neighbors to maximize the sum of their cumulative rewards while also satisfying each agent's safety constraints. We consider both peak and average constraints. In this scenario, there is no central controller coordinating the agents and both the rewards and constraints are only known to each agent locally/privately. We formulate the problem as a decentralized constrained multi-agent Markov Decision Problem and propose a momentum-based decentralized policy gradient method, DePAint, to solve it. To the best of our knowledge, this is the first privacy-preserving fully decentralized multi-agent reinforcement learning algorithm that considers both peak and average constraints. We then provide theoretical analysis and empirical evaluation of our algorithm in a number of scenarios and compare its performance to centralized algorithms that consider similar constraints.
Code (1)
Tasks
Multi-agent Reinforcement LearningPrivacy Preservingreinforcement-learningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MAD-PINN: A Decentralized Physics-Informed Machine Learning Framework for Safe and Optimal Multi-Agent Control
Co-optimizing safety and performance in large-scale multi-agent systems remains a fundamental challenge. Existing approaches based on multi-agent reinforcement learning (MARL), safety filtering, or Model Predictive Contr…
Multi-agent Reinforcement LearningDynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning
In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order to prevent dangerous situations from hap…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guarantee
The safety of decentralized reinforcement learning (RL) is a challenging problem since malicious agents can share their poisoned policies with benign agents. The paper investigates a cooperative backdoor attack in a dece…
Backdoor Attackreinforcement-learningReinforcement LearningReinforcement Learning (RL)Safety-Aware Multi-Agent Learning for Dynamic Network Bridging
Addressing complex cooperative tasks in safety-critical environments poses significant challenges for multi-agent systems, especially under conditions of partial observability. We focus on a dynamic network bridging task…
Collision AvoidanceMulti-agent Reinforcement LearningThink Smart, Act SMARL! Analyzing Probabilistic Logic Shields for Multi-Agent Reinforcement Learning
Safe reinforcement learning (RL) is crucial for real-world applications, and multi-agent interactions introduce additional safety challenges. While Probabilistic Logic Shields (PLS) has been a powerful proposal to enforc…
Multi-agent Reinforcement LearningQ-Learningreinforcement-learningReinforcement Learning+2