Inverse Reinforcement Learning from Non-Stationary Learning Agents

2024-10-18 · Kavinayan P. Sivakumar, Yi Shen, Zachary Bell, Scott Nivison, Boyuan Chen, Michael M. Zavlanos

In this paper, we study an inverse reinforcement learning problem that involves learning the reward function of a learning agent using trajectory data collected while this agent is learning its optimal policy. To address this problem, we propose an inverse reinforcement learning method that allows us to estimate the policy parameters of the learning agent which can then be used to estimate its reward function. Our method relies on a new variant of the behavior cloning algorithm, which we call bundle behavior cloning, and uses a small number of trajectories generated by the learning agent's policy at different points in time to learn a set of policies that match the distribution of actions observed in the sampled trajectories. We then use the cloned policies to train a neural network model that estimates the reward function of the learning agent. We provide a theoretical analysis to show a complexity result on bound guarantees for our method that beats standard behavior cloning as well as numerical experiments for a reinforcement learning problem that validate the proposed method.

📄 PDF Abstract BibTeX arXiv:2410.14135

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Adversarial recovery of agent rewards from latent spaces of the limit order book

2019-12-09 · Jacobo Roa-Vicens, Yuanbo Wang, Virgile Mison, Yarin Gal 외

Inverse reinforcement learning has proved its ability to explain state-action trajectories of expert agents by recovering their underlying reward functions in increasingly challenging environments. Recent advances in adv…

Reinforcement Learning

Langevin Dynamics for Adaptive Inverse Reinforcement Learning of Stochastic Gradient Algorithms

2020-06-20 · Vikram Krishnamurthy, George Yin

Inverse reinforcement learning (IRL) aims to estimate the reward function of optimizing agents by observing their response (estimates or actions). This paper considers IRL when noisy estimates of the gradient of a reward…

reinforcement-learning Reinforcement Learning (RL)

Multi-Agent Generative Adversarial Imitation Learning

2018-07-26 · NeurIPS 2018 12 · Jiaming Song, Hongyu Ren, Dorsa Sadigh, Stefano Ermon

Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence …

Imitation Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model Free Reinforcement Learning Algorithm for Stationary Mean field Equilibrium for Multiple Types of Agents

2020-12-31 · Arnob Ghosh, Vaneet Aggarwal

We consider a multi-agent Markov strategic interaction over an infinite horizon where agents can be of multiple types. We model the strategic interaction as a mean-field game in the asymptotic limit when the number of ag…

Reinforcement Learning (RL)

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

2020-10-31 · Dong-Ki Kim, Miao Liu, Matthew Riemer, Chuangchuang Sun 외

A fundamental challenge in multiagent reinforcement learning is to learn beneficial behaviors in a shared environment with other simultaneously learning agents. In particular, each agent perceives the environment as effe…