Inverse reinforcement learning for video games

2018-10-24 · Aaron Tucker, Adam Gleave, Stuart Russell

Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function. It is often easier to provide demonstrations of a target behavior than to design a reward function describing that behavior. Inverse reinforcement learning (IRL) algorithms can infer a reward from demonstrations in low-dimensional continuous control environments, but there has been little work on applying IRL to high-dimensional video games. In our CNN-AIRL baseline, we modify the state-of-the-art adversarial IRL (AIRL) algorithm to use CNNs for the generator and discriminator. To stabilize training, we normalize the reward and increase the size of the discriminator training dataset. We additionally learn a low-dimensional state representation using a novel autoencoder architecture tuned for video game environments. This embedding is used as input to the reward network, improving the sample efficiency of expert demonstrations. Our method achieves high-level performance on the simple Catcher video game, substantially outperforming the CNN-AIRL baseline. We also score points on the Enduro Atari racing game, but do not match expert performance, highlighting the need for further work.

📄 PDF Abstract BibTeX arXiv:1810.10593

Code (1)

HumanCompatibleAI/atari-irl 공식 구현 tf

Tasks

continuous-control Continuous ControlDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations

2018-01-07 · ICML 2018 7 · Xingyu Wang, Diego Klabjan

This paper considers the problem of inverse reinforcement learning in zero-sum stochastic games when expert demonstrations are known to be not optimal. Compared to previous works that decouple agents in the game by assum…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Inverse Concave-Utility Reinforcement Learning is Inverse Game Theory

2024-05-29 · Mustafa Mert Çelikok, Frans A. Oliehoek, Jan-Willem van de Meent

We consider inverse reinforcement learning problems with concave utilities. Concave Utility Reinforcement Learning (CURL) is a generalisation of the standard RL objective, which employs a concave function of the state oc…

Imitation Learning Offline RL reinforcement-learning Reinforcement Learning

Data Poisoning to Fake a Nash Equilibrium in Markov Games

2023-06-13 · Young Wu, Jeremy McMahan, Xiaojin Zhu, Qiaomin Xie

We characterize offline data poisoning attacks on Multi-Agent Reinforcement Learning (MARL), where an attacker may change a data set in an attempt to install a (potentially fictitious) unique Markov-perfect Nash equilibr…

Data Poisoning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Games

2014-03-25 · Xiaomin Lin, Peter A. Beling, Randy Cogill

The focus of this paper is a Bayesian framework for solving a class of problems termed multi-agent inverse reinforcement learning (MIRL). Compared to the well-known inverse reinforcement learning (IRL) problem, MIRL is f…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Valence Prediction

A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games

2025-09-03 · Zhengyang Li, Qijin Ji, Xinghong Ling, Quan Liu arxiv

Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated its application potential in modern games. Beginning with foundational work and progressing to landmark achievements such as AlphaStar in…

Multi-agent Reinforcement Learning Starcraft II