Boundary-Aware Cascade Networks for Temporal Action Segmentation
Identifying human action segments in an untrimmed video is still challenging due to boundary ambiguity and over-segmentation issues. To address these problems, we present a new boundary-aware cascade network by introducing two novel components. First, we devise a new cascading paradigm, called Stage Cascade, to enable our model to have adaptive receptive fields and more confident predictions for ambiguous frames. Second, we design a general and principled smoothing operation, termed as local barrier pooling, to aggregate local predictions by leveraging semantic boundary information. Moreover, these two components can be jointly fine-tuned in an end-to-end manner. We perform experiments on three challenging datasets: 50Salads, GTEA and Breakfast dataset, demonstrating that our framework significantly out-performs the current state-of-the-art methods. The code is available at https://github.com/MCG-NJU/BCN.
Code (1)
Tasks
Action SegmentationTemporal Action SegmentationSimilar Papers 제목 키워드 기반
Cascaded Boundary Regression for Temporal Action Detection
Temporal action detection in long videos is an important problem. State-of-the-art methods address this problem by applying action classifiers on sliding windows. Although sliding windows may contain an identifiable port…
Action DetectionregressionImage Segmentation by Cascaded Region Agglomeration
We propose a hierarchical segmentation algorithm that starts with a very fine oversegmentation and gradually merges regions using a cascade of boundary classifiers. This approach allows the weights of region and boundary…
Image SegmentationSegmentationSemantic SegmentationEfficient Temporal Action Segmentation via Boundary-aware Query Voting
Although the performance of Temporal Action Segmentation (TAS) has improved in recent years, achieving promising results often comes with a high computational cost due to dense inputs, complex model structures, and resou…
Action SegmentationInstance SegmentationSegmentationSemantic Segmentation+1IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correction is treated as an isolated edit, limit…
Action UnderstandingAction SegmentationDo we really need temporal convolutions in action segmentation?
Action classification has made great progress, but segmenting and recognizing actions from long untrimmed videos remains a challenging problem. Most state-of-the-art methods focus on designing temporal convolution-based …
Action ClassificationAction SegmentationInductive Bias