Scalable spectral representations for multi-agent reinforcement learning in network MDPs
Network Markov Decision Processes (MDPs), a popular model for multi-agent control, pose a significant challenge to efficient learning due to the exponential growth of the global state-action space with the number of agents. In this work, utilizing the exponential decay property of network dynamics, we first derive scalable spectral local representations for network MDPs, which induces a network linear subspace for the local $Q$-function of each agent. Building on these local spectral representations, we design a scalable algorithmic framework for continuous state-action network MDPs, and provide end-to-end guarantees for the convergence of our algorithm. Empirically, we validate the effectiveness of our scalable representation-based approach on two benchmark problems, and demonstrate the advantages of our approach over generic function approximation approaches to representing the local $Q$-functions.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-agent Reinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hidden Coalitions in Multi-Agent AI: A Spectral Diagnostic from Internal Representations
Collections of interacting AI agents can form coalitions, creating emergent group-level organization that is critical for AI safety and alignment. However, observing agent behavior alone is often insufficient to distingu…
Multi-agent Reinforcement LearningFew-shot Multispectral Segmentation with Representations Generated by Reinforcement Learning
The task of segmentation of multispectral images, which are images with numerous channels or bands, each capturing a specific range of wavelengths of electromagnetic radiation, has been previously explored in contexts wi…
Image Segmentationreinforcement-learningReinforcement LearningSegmentation+1On The Effect of Auxiliary Tasks on Representation Dynamics
While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achieved. This work develops our understandin…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Unified Framework for Locality in Scalable MARL
Scalable methods for networked multi-agent reinforcement learning let each agent plan using only a small neighborhood of the agent graph. This works only when the system is value-local, meaning a perturbation at one agen…
Multi-agent Reinforcement LearningPrism: Spectral Parameter Sharing for Multi-Agent Reinforcement Learning
Parameter sharing is a key strategy in multi-agent reinforcement learning (MARL) for improving scalability, yet conventional fully shared architectures often collapse into homogeneous behaviors. Recent methods introduce …
Multi-agent Reinforcement Learning