ColorGrid: A Multi-Agent Non-Stationary Environment for Goal Inference and Assistance
Autonomous agents' interactions with humans are increasingly focused on adapting to their changing preferences in order to improve assistance in real-world tasks. Effective agents must learn to accurately infer human goals, which are often hidden, to collaborate well. However, existing Multi-Agent Reinforcement Learning (MARL) environments lack the necessary attributes required to rigorously evaluate these agents' learning capabilities. To this end, we introduce ColorGrid, a novel MARL environment with customizable non-stationarity, asymmetry, and reward structure. We investigate the performance of Independent Proximal Policy Optimization (IPPO), a state-of-the-art (SOTA) MARL algorithm, in ColorGrid and find through extensive ablations that, particularly with simultaneous non-stationary and asymmetric goals between a `leader'' agent representing a human and a `follower'' assistant agent, ColorGrid is unsolved by IPPO. To support benchmarking future MARL algorithms, we release our environment code, model checkpoints, and trajectory visualizations at https://github.com/andreyrisukhin/ColorGrid.
Code (1)
Tasks
BenchmarkingMulti-agent Reinforcement LearningSimilar Papers 제목 키워드 기반
Causal Knowledge Transfer for Multi-Agent Reinforcement Learning in Dynamic Environments
[Context] Multi-agent reinforcement learning (MARL) has achieved notable success in environments where agents must learn coordinated behaviors. However, transferring knowledge across agents remains challenging in non-sta…
Multi-agent Reinforcement LearningOpen-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction
We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task agents. We first identify two main challe…
DiversityMinecraftRepresentation LearningZero-shot GeneralizationDistributed Online Life-Long Learning (DOL3) for Multi-agent Trust and Reputation Assessment in E-commerce
Trust and Reputation Assessment of service providers in citizen-focused environments like e-commerce is vital to maintain the integrity of the interactions among agents. The goals and objectives of both the service provi…
Agent Probing Interaction Policies
Reinforcement learning in a multi agent system is difficult because these systems are inherently non-stationary in nature. In such a case, identifying the type of the opposite agent is crucial and can help us address thi…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)A Hybrid, PDE-ODE Control Strategy for Intercepting an Intelligent, well-informed Target in a Stationary, Cluttered Environment
In [1,2] a new class of intelligent controllers that can semantically embed an agent in a spatial context constraining its behavior in a goal-oriented manner was suggested. A controller of such a class can guide an agent…