Defense Against Reward Poisoning Attacks in Reinforcement Learning

2021-02-10 · Kiarash Banihashem, Adish Singla, Goran Radanovic

We study defense strategies against reward poisoning attacks in reinforcement learning. As a threat model, we consider attacks that minimally alter rewards to make the attacker's target policy uniquely optimal under the poisoned rewards, with the optimality gap specified by an attack parameter. Our goal is to design agents that are robust against such attacks in terms of the worst-case utility w.r.t. the true, unpoisoned, rewards while computing their policies under the poisoned rewards. We propose an optimization framework for deriving optimal defense policies, both when the attack parameter is known and unknown. Moreover, we show that defense policies that are solutions to the proposed optimization problems have provable performance guarantees. In particular, we provide the following bounds with respect to the true, unpoisoned, rewards: a) lower bounds on the expected return of the defense policies, and b) upper bounds on how suboptimal these defense policies are compared to the attacker's target policy. We conclude the paper by illustrating the intuitions behind our formal results, and showing that the derived bounds are non-trivial.

📄 PDF Abstract BibTeX arXiv:2102.05776

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Adaptive Reward-Poisoning Attacks against Reinforcement Learning

2020-03-27 · ICML 2020 1 · Xuezhou Zhang, Yuzhe ma, Adish Singla, Xiaojin Zhu

In reward-poisoning attacks against reinforcement learning (RL), an attacker can perturb the environment reward $r_t$ into $r_t+\delta_t$ at each step, with the goal of forcing the RL agent to learn a nefarious policy. W…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep Partition Aggregation: Provable Defense against General Poisoning Attacks

2020-06-26 · Alexander Levine, Soheil Feizi

Adversarial poisoning attacks distort training data in order to corrupt the test-time behavior of a classifier. A provable defense provides a certificate for each test sample, which is a lower bound on the magnitude of a…

COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks

2022-03-16 · ICLR 2022 4 · Fan Wu, Linyi Li, Chejian Xu, huan zhang 외

As reinforcement learning (RL) has achieved near human-level performance in a variety of tasks, its robustness has raised great attention. While a vast body of research has explored test-time (evasion) attacks in RL and …

Offline RL reinforcement-learning Reinforcement Learning (RL)

Deep Partition Aggregation: Provable Defenses against General Poisoning Attacks

2021-01-01 · ICLR 2021 1 · Alexander Levine, Soheil Feizi

Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments

2021-02-16 · Amin Rakhsha, Xuezhou Zhang, Xiaojin Zhu, Adish Singla

We study black-box reward poisoning attacks against reinforcement learning (RL), in which an adversary aims to manipulate the rewards to mislead a sequence of RL agents with unknown algorithms to learn a nefarious policy…

reinforcement-learning Reinforcement Learning (RL)