A Novel Bifurcation Method for Observation Perturbation Attacks on Reinforcement Learning Agents: Load Altering Attacks on a Cyber Physical Power System
Components of cyber physical systems, which affect real-world processes, are often exposed to the internet. Replacing conventional control methods with Deep Reinforcement Learning (DRL) in energy systems is an active area of research, as these systems become increasingly complex with the advent of renewable energy sources and the desire to improve their efficiency. Artificial Neural Networks (ANN) are vulnerable to specific perturbations of their inputs or features, called adversarial examples. These perturbations are difficult to detect when properly regularized, but have significant effects on the ANN's output. Because DRL uses ANN to map optimal actions to observations, they are similarly vulnerable to adversarial examples. This work proposes a novel attack technique for continuous control using Group Difference Logits loss with a bifurcation layer. By combining aspects of targeted and untargeted attacks, the attack significantly increases the impact compared to an untargeted attack, with drastically smaller distortions than an optimally targeted attack. We demonstrate the impacts of powerful gradient-based attacks in a realistic smart energy environment, show how the impacts change with different DRL agents and training procedures, and use statistical and time-series analysis to evaluate attacks' stealth. The results show that adversarial attacks can have significant impacts on DRL controllers, and constraining an attack's perturbations makes it difficult to detect. However, certain DRL architectures are far more robust, and robust training methods can further reduce the impact.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlDeep Reinforcement LearningTime Series AnalysisSimilar Papers 제목 키워드 기반
Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantization
Recent studies reveal that well-performing reinforcement learning (RL) agents in training often lack resilience against adversarial perturbations during deployment. This highlights the importance of building a robust age…
Deep Reinforcement LearningQuantizationreinforcement-learningReinforcement Learning+1Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses
Deep reinforcement learning (DRL) is vulnerable to adversarial perturbations. Adversaries can mislead the policies of DRL agents by perturbing the state of the environment observed by the agents. Existing attacks are fea…
Atari GamesDeep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)CIVA: Critic-Induced Value-Subspace Attacks on Visual World-Model Agents
Visual world-model agents such as DreamerV3 act through a recurrent latent state rather than a single observation, which weakens frame-wise observation attacks and makes their perturbations vary sharply over time under a…
CAMA: Exploring Collusive Adversarial Attacks in c-MARL
Cooperative multi-agent reinforcement learning (c-MARL) has been widely deployed in real-world applications, such as social robots, embodied intelligence, UAV swarms, etc. Nevertheless, many adversarial attacks still exi…
Multi-agent Reinforcement LearningDaylight: Assessing Generalization Skills of Deep Reinforcement Learning Agents
Deep reinforcement learning algorithms have recently achieved significant success in learning high-performing policies from purely visual observations. The ability to perform end-to-end learning from raw high dimensional…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)