Efficiently Solving Mixed-Hierarchy Games with Quasi-Policy Approximations
Multi-robot coordination often exhibits hierarchical structure, with some robots' decisions depending on the planned behaviors of others. While game theory provides a principled framework for such interactions, existing solvers struggle to handle mixed information structures that combine simultaneous (Nash) and hierarchical (Stackelberg) decision-making. We study N-robot forest-structured mixed-hierarchy games, in which each robot acts as a Stackelberg leader over its subtree while robots in different branches interact via Nash equilibria. We derive the Karush-Kuhn-Tucker (KKT) first-order optimality conditions for this class of games and show that they involve increasingly high-order derivatives of robots' best-response policies as the hierarchy depth grows, rendering a direct solution intractable. To overcome this challenge, we introduce a quasi-policy approximation that removes higher-order policy derivatives and develop an inexact Newton method for efficiently solving the resulting approximated KKT systems. We prove local exponential convergence of the proposed algorithm for games with non-quadratic objectives and nonlinear constraints. The approach is implemented in a highly optimized Julia library (MixedHierarchyGames.jl) and evaluated in hardware and simulated multi-agent experiments, demonstrating real-time convergence for complex mixed-hierarchy information structures.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Provably convergent quasistatic dynamics for mean-field two-player zero-sum games
In this paper, we study the problem of finding mixed Nash equilibrium for mean-field two-player zero-sum games. Solving this problem requires optimizing over two probability distributions. We consider a quasistatic Wasse…
Multiplicative weights, equalizers, and P=PPAD
We show that, by using multiplicative weights in a game-theoretic thought experiment (and an important convexity result on the composition of multiplicative weights with the relative entropy function), a symmetric bimatr…
Partially Observable Multi-Agent Reinforcement Learning with Information Sharing
We study provable multi-agent reinforcement learning (RL) in the general framework of partially observable stochastic games (POSGs). To circumvent the known hardness results and the use of computationally intractable ora…
Computational EfficiencyMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Convergence of two-timescale gradient descent ascent dynamics: finite-dimensional and mean-field perspectives
The two-timescale gradient descent-ascent (GDA) is a canonical gradient algorithm designed to find Nash equilibria in min-max games. We analyze the two-timescale GDA by investigating the effects of learning rate ratios o…
A class of stochastic games and moving free boundary problems
In this paper we propose and analyze a class of $N$-player stochastic games that include finite fuel stochastic games as a special case. We first derive sufficient conditions for the Nash equilibrium (NE) in the form of …