Health-Informed Policy Gradients for Multi-Agent Reinforcement Learning
This paper proposes a definition of system health in the context of multiple agents optimizing a joint reward function. We use this definition as a credit assignment term in a policy gradient algorithm to distinguish the contributions of individual agents to the global reward. The health-informed credit assignment is then extended to a multi-agent variant of the proximal policy optimization algorithm and demonstrated on particle and multiwalker robot environments that have characteristics such as system health, risk-taking, semi-expendable agents, continuous action spaces, and partial observability. We show significant improvement in learning performance compared to policy gradient methods that do not perform multi-agent credit assignment.
Code (1)
Tasks
Multi-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration
Context engineering has emerged as a primary lever for improving AI systems without parameter updates. Recent work showing that textual gradients do not function as real gradients motivates treating automatic prompt opti…
Gradient Informed Proximal Policy Optimization
We introduce a novel policy learning method that integrates analytical gradients from differentiable environments with the Proximal Policy Optimization (PPO) algorithm. To incorporate analytical gradients into the PPO fr…
Almost Sure Convergence of Networked Policy Gradient over Time-Varying Networks in Markov Potential Games
We propose networked policy gradient play for solving Markov potential games including continuous action and state spaces. In the decentralized algorithm, agents sample their actions from parametrized and differentiable …
Difference Rewards Policy Gradients
Policy gradient methods have become one of the most popular classes of algorithms for multi-agent reinforcement learning. A key challenge, however, that is not addressed by many of these methods is multi-agent credit ass…
counterfactualMulti-agent Reinforcement LearningPolicy Gradient MethodsReinforcement Learning (RL)ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models
Large language models (LLMs) transcend passive generation and act as goal-directed agents by invoking external tools. Reinforcement learning (RL) offers a principled framework for optimizing these emergent tool-use polic…
Reinforcement Learning