Stealthy Imitation: Reward-guided Environment-free Policy Stealing
Deep reinforcement learning policies, which are integral to modern control systems, represent valuable intellectual property. The development of these policies demands considerable resources, such as domain expertise, simulation fidelity, and real-world validation. These policies are potentially vulnerable to model stealing attacks, which aim to replicate their functionality using only black-box access. In this paper, we propose Stealthy Imitation, the first attack designed to steal policies without access to the environment or knowledge of the input range. This setup has not been considered by previous model stealing methods. Lacking access to the victim's input states distribution, Stealthy Imitation fits a reward model that allows to approximate it. We show that the victim policy is harder to imitate when the distribution of the attack queries matches that of the victim. We evaluate our approach across diverse, high-dimensional control tasks and consistently outperform prior data-free approaches adapted for policy stealing. Lastly, we propose a countermeasure that significantly diminishes the effectiveness of the attack.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningSimilar Papers 제목 키워드 기반
Stealthy Adversarial Attacks on Stochastic Multi-Armed Bandits
Adversarial attacks against stochastic multi-armed bandit (MAB) algorithms have been extensively studied in the literature. In this work, we focus on reward poisoning attacks and find most existing attacks can be easily …
Multi-Armed BanditsTowards a Reward-Free Reinforcement Learning Framework for Vehicle Control
Reinforcement learning plays a crucial role in vehicle control by guiding agents to learn optimal control strategies through designing or learning appropriate reward signals. However, in vehicle control applications, rew…
Imitation Learningreinforcement-learningReinforcement LearningPolicy Teaching in Reinforcement Learning via Environment Poisoning Attacks
We study a security threat to reinforcement learning where an attacker poisons the learning environment to force the agent into executing a target policy chosen by the attacker. As a victim, we consider RL agents whose o…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning
We study a security threat to reinforcement learning where an attacker poisons the learning environment to force the agent into executing a target policy chosen by the attacker. As a victim, we consider RL agents whose o…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards
The application of reinforcement learning (RL) in robotic control is still limited in the environments with sparse and delayed rewards. In this paper, we propose a practical self-imitation learning method named Self-Imit…
Imitation LearningMuJoCoReinforcement Learning (RL)