Convex-Concave Zero-Sum Stochastic Stackelberg Games
Zero-sum stochastic Stackelberg games can be used to model a large class of problems, ranging from economics to human robot interaction. In this paper, we develop policy gradient methods to solve these games from noisy gradient estimates computed from observed trajectories of play. We prove that our algorithms converge to Stackelberg equilibrium in polynomial time when the games are convex-concave. We also prove that reach-avoid problems are naturally modeled as convex-concave zero-sum stochastic Stackelberg games. Finally, we run experiments which demonstrate that modeling reach-avoid problems as Stackelberg games leads to solutions which are safer, thus less likely to result in collisions, and liver, thus more likely to reach their goals, than alternative solutions, in particular Nash equilibrium.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Convex-Concave Min-Max Stackelberg Games
Min-max optimization problems (i.e., min-max games) have been attracting a great deal of attention because of their applicability to a wide range of machine learning problems. Although significant progress has been made …
Solving Min-Max Optimization with Hidden Structure via Gradient Descent Ascent
Many recent AI architectures are inspired by zero-sum games, however, the behavior of their dynamics is still not well understood. Inspired by this, we study standard gradient descent ascent (GDA) dynamics in a specific …
Approachability in Stackelberg Stochastic Games with Vector Costs
The notion of approachability was introduced by Blackwell [1] in the context of vector-valued repeated games. The famous Blackwell's approachability theorem prescribes a strategy for approachability, i.e., for `steering'…
Decision MakingReinforcement LearningImplicit Learning Dynamics in Stackelberg Games: Equilibria Characterization, Convergence Analysis, and Empirical Study
Contemporary work on learning in continuous games has commonly overlooked the hierarchical decision-making structure present in machine learning problems formulated as games, instead treating them as simultaneous play ga…
Decision MakingExponential Convergence of Gradient Methods in Concave Network Zero-sum Games
Motivated by Generative Adversarial Networks, we study the computation of Nash equilibrium in concave network zero-sum games (NZSGs), a multiplayer generalization of two-player zero-sum games first proposed with linear p…