Inverse-Inverse Reinforcement Learning. How to Hide Strategy from an Adversarial Inverse Reinforcement Learner
Inverse reinforcement learning (IRL) deals with estimating an agent's utility function from its actions. In this paper, we consider how an agent can hide its strategy and mitigate an adversarial IRL attack; we call this inverse IRL (I-IRL). How should the decision maker choose its response to ensure a poor reconstruction of its strategy by an adversary performing IRL to estimate the agent's strategy? This paper comprises four results: First, we present an adversarial IRL algorithm that estimates the agent's strategy while controlling the agent's utility function. Our second result for I-IRL result spoofs the IRL algorithm used by the adversary. Our I-IRL results are based on revealed preference theory in micro-economics. The key idea is for the agent to deliberately choose sub-optimal responses that sufficiently masks its true strategy. Third, we give a sample complexity result for our main I-IRL result when the agent has noisy estimates of the adversary specified utility function. Finally, we illustrate our I-IRL scheme in a radar problem where a meta-cognitive radar is trying to mitigate an adversarial target.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement Learning (RL)Similar Papers 제목 키워드 기반
How can a Radar Mask its Cognition?
A cognitive radar is a constrained utility maximizer that adapts its sensing mode in response to a changing environment. If an adversary can estimate the utility function of a cognitive radar, it can determine the radar'…
Inverse Reinforcement Learning for Strategy Identification
In adversarial environments, one side could gain an advantage by identifying the opponent's strategy. For example, in combat games, if an opponents strategy is identified as overly aggressive, one could lay a trap that e…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Adversarial Radar Inference. From Inverse Tracking to Inverse Reinforcement Learning of Cognitive Radar
Cognitive sensing refers to a reconfigurable sensor that dynamically adapts its sensing mechanism by using stochastic control to optimize its sensing resources. For example, cognitive radars are sophisticated dynamical s…
Reinforcement Learning (RL)Stochastic OptimizationMeta-Cognition. An Inverse-Inverse Reinforcement Learning Approach for Cognitive Radars
This paper considers meta-cognitive radars in an adversarial setting. A cognitive radar optimally adapts its waveform (response) in response to maneuvers (probes) of a possibly adversarial moving target. A meta-cognitive…
reinforcement-learningReinforcement Learning (RL)Adversarial Exploration Strategy for Self-Supervised Imitation Learning
We present an adversarial exploration strategy, a simple yet effective imitation learning scheme that incentivizes exploration of an environment without any extrinsic reward or human demonstration. Our framework consists…
Deep Reinforcement LearningImitation LearningOpenAI GymReinforcement Learning