Reward-Preserving Attacks For Robust Reinforcement Learning
Adversarial training in reinforcement learning (RL) is challenging because perturbations cascade through trajectories and compound over time, making fixed-strength attacks either overly destructive or too conservative. We propose reward-preserving attacks, which adapt adversarial strength so that an $α$ fraction of the nominal-to-worst-case return gap remains achievable at each state. In deep RL, perturbation magnitudes $η$ are selected dynamically, using a learned critic $Q((s,a),η)$ that estimates the expected return of $α$-reward-preserving rollouts. For intermediate values of $α$, this adaptive training yields policies that are robust across a wide range of perturbation magnitudes while preserving nominal performance, outperforming fixed-radius and uniformly sampled-radius adversarial training.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Reward Delay Attacks on Deep Reinforcement Learning
Most reinforcement learning algorithms implicitly assume strong synchrony. We present novel attacks targeting Q-learning that exploit a vulnerability entailed by this assumption by delaying the reward signal for a limite…
Deep Reinforcement LearningQ-Learningreinforcement-learningReinforcement Learning+1Adaptive Reward-Poisoning Attacks against Reinforcement Learning
In reward-poisoning attacks against reinforcement learning (RL), an attacker can perturb the environment reward $r_t$ into $r_t+\delta_t$ at each step, with the goal of forcing the RL agent to learn a nefarious policy. W…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Efficient Reward Poisoning Attacks on Online Deep Reinforcement Learning
We study reward poisoning attacks on online deep reinforcement learning (DRL), where the attacker is oblivious to the learning algorithm used by the agent and the dynamics of the environment. We demonstrate the intrinsic…
Data PoisoningDeep Reinforcement LearningMuJoCoreinforcement-learning+2Reinforcement Learning-Based Black-Box Model Inversion Attacks
Model inversion attacks are a type of privacy attack that reconstructs private data used to train a machine learning model, solely by accessing the model. Recently, white-box model inversion attacks leveraging Generative…
modelPrivacy Preservingreinforcement-learningReinforcement LearningUnderstanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning
To understand the security threats to reinforcement learning (RL) algorithms, this paper studies poisoning attacks to manipulate \emph{any} order-optimal learning algorithm towards a targeted policy in episodic RL and ex…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)