Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentation
Temporal action segmentation classifies the action of each frame in (long) video sequences. Due to the high cost of frame-wise labeling, we propose the first semi-supervised method for temporal action segmentation. Our method hinges on unsupervised representation learning, which, for temporal action segmentation, poses unique challenges. Actions in untrimmed videos vary in length and have unknown labels and start/end times. Ordering of actions across videos may also vary. We propose a novel way to learn frame-wise representations from temporal convolutional networks (TCNs) by clustering input features with added time-proximity condition and multi-resolution similarity. By merging representation learning with conventional supervised learning, we develop an "Iterative-Contrast-Classify (ICC)" semi-supervised learning scheme. With more labelled data, ICC progressively improves in performance; ICC semi-supervised learning, with 40% labelled videos, performs similar to fully-supervised counterparts. Our ICC improves MoF by {+1.8, +5.6, +2.5}% on Breakfast, 50Salads and GTEA respectively for 100% labelled videos.
Code (1)
Tasks
Action SegmentationRepresentation LearningSegmentationSemi-Supervised Video ClassificationTemporal Action SegmentationSimilar Papers 제목 키워드 기반
C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation
Temporal action segmentation tags action labels for every frame in an input untrimmed video containing multiple actions in a sequence. For the task of temporal action segmentation, we propose an encoder-decoder-style arc…
Action SegmentationDecoderRepresentation LearningSegmentation+1CRL+: A Novel Semi-Supervised Deep Active Contrastive Representation Learning-Based Text Classification Model for Insurance Data
Financial sector and especially the insurance industry collect vast volumes of text on a daily basis and through multiple channels (their agents, customer care centers, emails, social networks, and web in general). The i…
Active LearningRepresentation Learningtext-classificationText ClassificationSemi-supervised Time Series Classification by Temporal Relation Prediction
Semi-supervised learning (SSL) has proven to be a powerful algorithm in different domains by leveraging unlabeled data to mitigate the reliance on the tremendous annotated data. However, few efforts consider the underlyi…
ClassificationPredictionRelationRelation Prediction+4ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast
Medical data often exhibits long-tail distributions with heavy class imbalance, which naturally leads to difficulty in classifying the minority classes (i.e., boundary regions or rare objects). Recent work has significan…
Contrastive LearningImage SegmentationMedical Image SegmentationSemantic Segmentation+1ContraCluster: Learning to Classify without Labels by Contrastive Self-Supervision and Prototype-Based Semi-Supervision
The recent advances in representation learning inspire us to take on the challenging problem of unsupervised image classification tasks in a principled way. We propose ContraCluster, an unsupervised image classification …
Contrastive Learningimage-classificationImage ClassificationRepresentation Learning+2