VideoMix: Rethinking Data Augmentation for Video Classification
State-of-the-art video action classifiers often suffer from overfitting. They tend to be biased towards specific objects and scene cues, rather than the foreground action content, leading to sub-optimal generalization performances. Recent data augmentation strategies have been reported to address the overfitting problems in static image classifiers. Despite the effectiveness on the static image classifiers, data augmentation has rarely been studied for videos. For the first time in the field, we systematically analyze the efficacy of various data augmentation strategies on the video classification task. We then propose a powerful augmentation strategy VideoMix. VideoMix creates a new training video by inserting a video cuboid into another video. The ground truth labels are mixed proportionally to the number of voxels from each video. We show that VideoMix lets a model learn beyond the object and scene biases and extract more robust cues for action recognition. VideoMix consistently outperforms other augmentation baselines on Kinetics and the challenging Something-Something-V2 benchmarks. It also improves the weakly-supervised action localization performance on THUMOS'14. VideoMix pretrained models exhibit improved accuracies on the video detection task (AVA).
Code (3)
Tasks
Action LocalizationAction RecognitionClassificationData AugmentationGeneral ClassificationVideo ClassificationWeakly Supervised Action LocalizationSimilar Papers 제목 키워드 기반
ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognition
In this paper, we propose a data augmentation method for action recognition using instance segmentation. Although many data augmentation methods have been proposed for image recognition, few of them are tailored for acti…
Action RecognitionData AugmentationInstance SegmentationSegmentation+1VideoMix: Aggregating How-To Videos for Task-Oriented Learning
Tutorial videos are a valuable resource for people looking to learn new tasks. People often learn these skills by viewing multiple tutorial videos to get an overall understanding of a task by looking at different approac…
NavigateRethinking Cost-sensitive Classification in Deep Learning via Adversarial Data Augmentation
Cost-sensitive classification is critical in applications where misclassification errors widely vary in cost. However, over-parameterization poses fundamental challenges to the cost-sensitive modeling of deep neural netw…
Data AugmentationSoftEDA: Rethinking Rule-Based Data Augmentation with Soft Labels
Rule-based text data augmentation is widely used for NLP tasks due to its simplicity. However, this method can potentially damage the original meaning of the text, ultimately hurting the performance of the model. To over…
Data AugmentationRethinking Ultrasound Augmentation: A Physics-Inspired Approach
Medical Ultrasound (US), despite its wide use, is characterized by artifacts and operator dependency. Those attributes hinder the gathering and utilization of US datasets for the training of Deep Neural Networks used for…
Data Augmentation