Opponent Modeling in Deep Reinforcement Learning

2016-09-18 · He He, Jordan Boyd-Graber, Kevin Kwok, Hal Daumé III

Opponent modeling is necessary in multi-agent settings where secondary agents with competing goals also adapt their strategies, yet it remains challenging because strategies interact with each other and change. Most previous work focuses on developing probabilistic models or parameterized strategies for specific applications. Inspired by the recent success of deep reinforcement learning, we present neural-based models that jointly learn a policy and the behavior of opponents. Instead of explicitly predicting the opponent's action, we encode observation of the opponents into a deep Q-Network (DQN); however, we retain explicit modeling (if desired) using multitasking. By using a Mixture-of-Experts architecture, our model automatically discovers different strategy patterns of opponents without extra supervision. We evaluate our models on a simulated soccer game and a popular trivia game, showing superior performance over DQN and its variants.

📄 PDF Abstract BibTeX arXiv:1609.05559

Code (1)

hhexiy/opponent 공식 구현 torch

Tasks

Deep Reinforcement LearningMixture-of-Expertsreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Generalized Intention Modeling in Multi-Agent Reinforcement Learning

2026-05-29 · Mateusz Odrowaz-Sypniewski, Jasmine Bayrooti, Ajay Shankar, Amanda Prorok arxiv

Modeling an opponent's intent is critical for effective decision-making in non-cooperative, competitive, and general-sum multi-agent reinforcement learning. Existing opponent modeling methods encode intent using an embed…

Multi-agent Reinforcement Learning

Variational Autoencoders for Opponent Modeling in Multi-Agent Systems

2020-01-29 · Georgios Papoudakis, Stefano V. Albrecht

Multi-agent systems exhibit complex behaviors that emanate from the interactions of multiple agents in a shared environment. In this work, we are interested in controlling one agent in a multi-agent system and successful…

Reinforcement Learning

Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games

2019-09-18 · Macheng Shen, Jonathan P. How

This paper presents an algorithmic framework for learning robust policies in asymmetric imperfect-information games, where the joint reward could depend on the uncertain opponent type (a private information known only to…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Stochastic Optimization

Combining Deep Reinforcement Learning and Search with Generative Models for Game-Theoretic Opponent Modeling

2023-02-01 · Zun Li, Marc Lanctot, Kevin R. McKee, Luke Marris 외

Opponent modeling methods typically involve two crucial steps: building a belief distribution over opponents' strategies, and exploiting this opponent model by playing a best response. However, existing approaches typica…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Proximal Learning With Opponent-Learning Awareness

2022-10-18 · Stephen Zhao, Chris Lu, Roger Baker Grosse, Jakob Nicolaus Foerster

Learning With Opponent-Learning Awareness (LOLA) (Foerster et al. [2018a]) is a multi-agent reinforcement learning algorithm that typically learns reciprocity-based cooperation in partially competitive environments. Howe…