Iterate & Cluster: Iterative Semi-Supervised Action Recognition
We propose a novel system for active semi-supervised feature-based action recognition. Given time sequences of features tracked during movements our system clusters the sequences into actions. Our system is based on encoder-decoder unsupervised methods shown to perform clustering by self-organization of their latent representation through the auto-regression task. These methods were tested on human action recognition benchmarks and outperformed non-feature based unsupervised methods and achieved comparable accuracy to skeleton-based supervised methods. However, such methods rely on K-Nearest Neighbours (KNN) associating sequences to actions, and general features with no annotated data would correspond to approximate clusters which could be further enhanced. Our system proposes an iterative semi-supervised method to address this challenge and to actively learn the association of clusters and actions. The method utilizes latent space embedding and clustering of the unsupervised encoder-decoder to guide the selection of sequences to be annotated in each iteration. Each iteration, the selection aims to enhance action recognition accuracy while choosing a small number of sequences for annotation. We test the approach on human skeleton-based action recognition benchmarks assuming that only annotations chosen by our method are available and on mouse movements videos recorded in lab experiments. We show that our system can boost recognition performance with only a small percentage of annotations. The system can be used as an interactive annotation tool to guide labeling efforts for 'in the wild' videos of various objects and actions to reach robust recognition.
Code (1)
Tasks
Action RecognitionClusteringDecoderSkeleton Based Action RecognitionTemporal Action LocalizationSimilar Papers 제목 키워드 기반
Clustering with Semidefinite Programming and Fixed Point Iteration
We introduce a novel method for clustering using a semidefinite programming (SDP) relaxation of the Max k-Cut problem. The approach is based on a new methodology for rounding the solution of an SDP relaxation using itera…
ClusteringUnsupervised Clustering using Pseudo-semi-supervised Learning
In this paper, we propose a framework that leverages semi-supervised models to improve unsupervised clustering performance. To leverage semi-supervised models, we first need to automatically generate labels, called pseud…
ClusteringIterative Contrast-Classify For Semi-supervised Temporal Action Segmentation
Temporal action segmentation classifies the action of each frame in (long) video sequences. Due to the high cost of frame-wise labeling, we propose the first semi-supervised method for temporal action segmentation. Our m…
Action SegmentationRepresentation LearningSegmentationSemi-Supervised Video Classification+1Structured Graph Learning for Clustering and Semi-supervised Classification
Graphs have become increasingly popular in modeling structures and interactions in a wide variety of problems during the last decade. Graph-based clustering and semi-supervised classification techniques have shown impres…
ClassificationClusteringGeneral ClassificationGraph LearningSemi-Supervised Nonlinear Distance Metric Learning via Forests of Max-Margin Cluster Hierarchies
Metric learning is a key problem for many data mining and machine learning applications, and has long been dominated by Mahalanobis methods. Recent advances in nonlinear metric learning have demonstrated the potential po…
ClusteringImage RetrievalMetric LearningRetrieval