Contrastive Video Representation Learning via Adversarial Perturbations
Adversarial perturbations are noise-like patterns that can subtly change the data, while failing an otherwise accurate classifier. In this paper, we propose to use such perturbations within a novel contrastive learning setup to build negative samples, which are then used to produce improved video representations. To this end, given a well-trained deep model for per-frame video recognition, we first generate adversarial noise adapted to this model. Positive and negative bags are produced using the original data features from the full video sequence and their perturbed counterparts, respectively. Unlike the classic contrastive learning methods, we develop a binary classification problem that learns a set of discriminative hyperplanes -- as a subspace -- that will separate the two bags from each other. This subspace is then used as a descriptor for the video, dubbed \emph{discriminative subspace pooling}. As the perturbed features belong to data classes that are likely to be confused with the original features, the discriminative subspace will characterize parts of the feature space that are more representative of the original data, and thus may provide robust video representations. To learn such descriptors, we formulate a subspace learning objective on the Stiefel manifold and resort to Riemannian optimization methods for solving it efficiently. We provide experiments on several video datasets and demonstrate state-of-the-art results.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionBinary ClassificationContrastive LearningRepresentation LearningRiemannian optimizationVideo RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility
Contrastive self-supervised learning (CSL) has managed to match or surpass the performance of supervised learning in image and video classification. However, it is still largely unknown if the nature of the representatio…
Adversarial RobustnessSelf-Supervised LearningSensitivityVideo ClassificationRobust Pre-Training by Adversarial Contrastive Learning
Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness In this work, we improve robustness-aware self-supervised pre-training by learni…
Adversarial RobustnessContrastive LearningC-LEAD: Contrastive Learning for Enhanced Adversarial Defense
Deep neural networks (DNNs) have achieved remarkable success in computer vision tasks such as image classification, segmentation, and object detection. However, they are vulnerable to adversarial attacks, which can cause…
Adversarial RobustnessContrastive LearningImage ClassificationAdversarial DefenseReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learning
While Goal-Conditioned Reinforcement Learning (GCRL) has gained attention, its algorithmic robustness against adversarial perturbations remains unexplored. The attacks and robust representation training methods that are …
Adversarial Robustnessreinforcement-learningReinforcement LearningFrom Pretrain to Pain: Adversarial Vulnerability of Video Foundation Models Without Task Knowledge
Large-scale Video Foundation Models (VFMs) has significantly advanced various video-related tasks, either through task-specific models or Multi-modal Large Language Models (MLLMs). However, the open accessibility of VFMs…
Contrastive LearningAdversarial Attack