MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation
Temporally locating and classifying action segments in long untrimmed videos is of particular interest to many applications like surveillance and robotics. While traditional approaches follow a two-step pipeline, by generating frame-wise probabilities and then feeding them to high-level temporal models, recent approaches use temporal convolutions to directly classify the video frames. In this paper, we introduce a multi-stage architecture for the temporal action segmentation task. Each stage features a set of dilated temporal convolutions to generate an initial prediction that is refined by the next one. This architecture is trained using a combination of a classification loss and a proposed smoothing loss that penalizes over-segmentation errors. Extensive evaluation shows the effectiveness of the proposed model in capturing long-range dependencies and recognizing action segments. Our model achieves state-of-the-art results on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.
Code (2)
Tasks
Action SegmentationSegmentationTemporal Action SegmentationSimilar Papers 제목 키워드 기반
Skeleton-Based Action Segmentation with Multi-Stage Spatial-Temporal Graph Convolutional Neural Networks
The ability to identify and temporally segment fine-grained actions in motion capture sequences is crucial for applications in human movement analysis. Motion capture is typically performed with optical or inertial measu…
Action SegmentationSkeleton Based Action SegmentationTime SeriesTime Series AnalysisMS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation
With the success of deep learning in classifying short trimmed videos, more attention has been focused on temporally segmenting and classifying activities in long untrimmed videos. State-of-the-art approaches for action …
Action SegmentationSegmentationTemporal Action SegmentationAutomated freezing of gait assessment with marker-based motion capture and multi-stage spatial-temporal graph convolutional neural networks
Freezing of gait (FOG) is a common and debilitating gait impairment in Parkinson's disease. Further insight into this phenomenon is hampered by the difficulty to objectively assess FOG. To meet this clinical need, this p…
Action SegmentationSegmentationMS-TCRNet: Multi-Stage Temporal Convolutional Recurrent Networks for Action Segmentation Using Sensor-Augmented Kinematics
Action segmentation is a challenging task in high-level process analysis, typically performed on video or kinematic data obtained from various sensors. This work presents two contributions related to action segmentation …
Action SegmentationData AugmentationSegmentationTricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation
Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years. It is typically being modeled as a sequence labeling problem b…
Action SegmentationDecoderSegmentation