Fooling Detection Alone is Not Enough: First Adversarial Attack against Multiple Object Tracking
Recent work in adversarial machine learning started to focus on the visual perception in autonomous driving and studied Adversarial Examples (AEs) for object detection models. However, in such visual perception pipeline the detected objects must also be tracked, in a process called Multiple Object Tracking (MOT), to build the moving trajectories of surrounding obstacles. Since MOT is designed to be robust against errors in object detection, it poses a general challenge to existing attack techniques that blindly target objection detection: we find that a success rate of over 98% is needed for them to actually affect the tracking results, a requirement that no existing attack technique can satisfy. In this paper, we are the first to study adversarial machine learning attacks against the complete visual perception pipeline in autonomous driving, and discover a novel attack technique, tracker hijacking, that can effectively fool MOT using AEs on object detection. Using our technique, successful AEs on as few as one single frame can move an existing object in to or out of the headway of an autonomous vehicle to cause potential safety hazards. We perform evaluation using the Berkeley Deep Drive dataset and find that on average when 3 frames are attacked, our attack can have a nearly 100% success rate while attacks that blindly target object detection only have up to 25%.
Code (1)
Tasks
Adversarial AttackAutonomous DrivingBIG-bench Machine LearningMultiple Object TrackingObjectobject-detectionObject DetectionObject TrackingSimilar Papers 제목 키워드 기반
Fooling Detection Alone is Not Enough: Adversarial Attack against Multiple Object Tracking
Recent work in adversarial machine learning started to focus on the visual perception in autonomous driving and studied Adversarial Examples (AEs) for object detection models. However, in such visual perception pipeline …
Adversarial AttackAutonomous DrivingBIG-bench Machine LearningMultiple Object Tracking+4A New Kind of Adversarial Example
Almost all adversarial attacks are formulated to add an imperceptible perturbation to an image in order to fool a model. Here, we consider the opposite which is adversarial examples that can fool a human but not a model.…
Evolutionary AlgorithmsPlaying Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind
As large language models (LLMs) become the engine behind conversational systems, their ability to reason about the intentions and states of their dialogue partners (i.e., form and use a theory-of-mind, or ToM) becomes in…
Reinforcement LearningAdversarial Profiles: Detecting Out-Distribution & Adversarial Samples in Pre-trained CNNs
Despite high accuracy of Convolutional Neural Networks (CNNs), they are vulnerable to adversarial and out-distribution examples. There are many proposed methods that tend to detect or make CNNs robust against these fooli…
Adversarial AttackRobust Android Malware Detection System against Adversarial Attacks using Q-Learning
The current state-of-the-art Android malware detection systems are based on machine learning and deep learning models. Despite having superior performance, these models are susceptible to adversarial attacks. Therefore i…
Adversarial DefenseAndroid Malware DetectionBIG-bench Machine LearningMalware Detection+4