Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems

2020-07-07 · Kunal Pattanayak, Vikram Krishnamurthy

This paper presents an inverse reinforcement learning~(IRL) framework for Bayesian stopping time problems. By observing the actions of a Bayesian decision maker, we provide a necessary and sufficient condition to identify if these actions are consistent with optimizing a cost function. In a Bayesian (partially observed) setting, the inverse learner can at best identify optimality wrt the observed strategies. Our IRL algorithm identifies optimality and then constructs set-valued estimates of the cost function.To achieve this IRL objective, we use novel ideas from Bayesian revealed preferences stemming from microeconomics. We illustrate the proposed IRL scheme using two important examples of stopping time problems, namely, sequential hypothesis testing and Bayesian search. As a real-world example, we illustrate using a YouTube dataset comprising metadata from 190000 videos how the proposed IRL method predicts user engagement in online multimedia platforms with high accuracy. Finally, for finite datasets, we propose an IRL detection algorithm and give finite sample bounds on its error probabilities.

📄 PDF Abstract BibTeX arXiv:2007.03481

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning (RL)Two-sample testing

Identifiability in inverse reinforcement learning

2021-06-07 · NeurIPS 2021 12 · Haoyang Cao, Samuel N. Cohen, Lukasz Szpruch

Inverse reinforcement learning attempts to reconstruct the reward function in a Markov decision problem, using observations of agent actions. As already observed in Russell [1998] the problem is ill-posed, and the reward…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games

2018-06-26 · Xiaomin Lin, Stephen C. Adams, Peter A. Beling

This paper addresses the problem of multi-agent inverse reinforcement learning (MIRL) in a two-player general-sum stochastic game framework. Five variants of MIRL are considered: uCS-MIRL, advE-MIRL, cooE-MIRL, uCE-MIRL,…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Revealed Bayesian Persuasion

2025-04-02 · Jeffrey Mensch

When is random choice generated by a decision maker (DM) who is Bayesian-persuaded by a sender? In this paper, I consider a DM whose state-dependent preferences are known to an analyst, yet chooses stochastically as a fu…

Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification

2024-03-11 · Joar Skalse, Alessandro Abate

Inverse reinforcement learning (IRL) aims to infer an agent's preferences (represented as a reward function $R$) from their behaviour (represented as a policy $\pi$). To do this, we need a behavioural model of how $\pi$ …

reinforcement-learning Reinforcement Learning Sensitivity

Inversion of Bayesian Networks

2022-12-20 · Jesse van Oostrum, Peter van Hintum, Nihat Ay

Variational autoencoders and Helmholtz machines use a recognition network (encoder) to approximate the posterior distribution of a generative model (decoder). In this paper we study the necessary and sufficient propertie…

Decoder