Non-Robust Feature Mapping in Deep Reinforcement Learning
Adversarial perturbations to state observations can dramatically degrade the performance of deep reinforcement learning policies, and thus raise concerns regarding the robustness of deep reinforcement learning agents. A sizeable body of work has focused on addressing the robustness problem in deep reinforcement learning, and there are several recent proposals for adversarial training methods in the deep reinforcement learning domain. In our work we focus on the robustness of state-of-the-art adversarially trained deep reinforcement learning policies and vanilla trained deep reinforcement learning polices. We propose two novel algorithms to map non-robust features in deep reinforcement learning policies. We conduct several experiments in the Arcade Learning Environment (ALE), and with our proposed feature mapping algorithms we show that while the state-of-the-art adversarial training method eliminates a certain set of non-robust features, a new set of non-robust features more intrinsic to the adversarial training are created. Our results lay out concerns that arise when using existing state-of-the-art adversarial training methods, and we believe our proposed feature mapping algorithm can aid in the process of building more robust deep reinforcement learning policies.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesDeep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mapping
Modern tasks in reinforcement learning have large state and action spaces. To deal with them efficiently, one often uses predefined feature mapping to represent states and actions in a low-dimensional space. In this pape…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Nonparametric Bellman Mappings for Reinforcement Learning: Application to Robust Adaptive Filtering
This paper designs novel nonparametric Bellman mappings in reproducing kernel Hilbert spaces (RKHSs) for reinforcement learning (RL). The proposed mappings benefit from the rich approximating properties of RKHSs, adopt n…
Dimensionality ReductionReinforcement Learning (RL)Integrating Deep Reinforcement and Supervised Learning to Expedite Indoor Mapping
The challenge of mapping indoor environments is addressed. Typical heuristic algorithms for solving the motion planning problem are frontier-based methods, that are especially effective when the environment is completely…
Decision MakingDeep Reinforcement LearningMotion PlanningNearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes
We study reinforcement learning (RL) with linear function approximation. For episodic time-inhomogeneous linear Markov decision processes (linear MDPs) whose transition probability can be parameterized as a linear functi…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Learning objects from pixels
We show how discrete objects can be learnt in an unsupervised fashion from pixels, and how to perform reinforcement learning using this object representation. More precisely, we construct a differentiable mapping from a…
Atari GamesObjectPositionreinforcement-learning+2