Temporal Deformable Residual Networks for Action Segmentation in Videos
This paper is about temporal segmentation of human actions in videos. We introduce a new model -- temporal deformable residual network (TDRN) -- aimed at analyzing video intervals at multiple temporal scales for labeling video frames. Our TDRN computes two parallel temporal streams: i) Residual stream that analyzes video information at its full temporal resolution, and ii) Pooling/unpooling stream that captures long-range video information at different scales. The former facilitates local, fine-scale action segmentation, and the latter uses multiscale context for improving accuracy of frame classification. These two streams are computed by a set of temporal residual modules with deformable convolutions, and fused by temporal residuals at the full video resolution. Our evaluation on the University of Dundee 50 Salads, Georgia Tech Egocentric Activities, and JHU-ISI Gesture and Skill Assessment Working Set demonstrates that TDRN outperforms the state of the art in frame-wise segmentation accuracy, segmental edit score, and segmental overlap F1 score.
Code (0)
등록된 구현이 없습니다.
Tasks
Action SegmentationSegmentationSimilar Papers 제목 키워드 기반
Deformable Tube Network for Action Detection in Videos
We address the problem of spatio-temporal action detection in videos. Existing methods commonly either ignore temporal context in action recognition and localization, or lack the modelling of flexible shapes of action tu…
Action DetectionAction RecognitionA deep deformable residual learning network for SAR images segmentation
Reliable automatic target segmentation in Synthetic Aperture Radar (SAR) imagery has played an important role in the SAR fields. Different from the traditional methods, Spectral Residual (SR) and CFAR detector, with the …
Learning TheorySegmentationDeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos
World models for deformable objects should recover not only geometry and appearance, but also underlying physical dynamics, interaction grounding, and material behavior. Learning such a model from real videos is challeng…
D2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos
Despite receiving significant attention from the research community, the task of segmenting and tracking objects in monocular videos still has much room for improvement. Existing works have simultaneously justified the e…
Multi-Object Tracking and SegmentationSegmentationSemantic SegmentationUnsupervised Video Object Segmentation+4D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos
Despite receiving significant attention from the research community, the task of segmenting and tracking objects in monocular videos still has much room for improvement. Existing works have simultaneously justified the e…
SegmentationSemantic SegmentationUnsupervised Video Object SegmentationVideo Object Segmentation+2