An end-to-end generative framework for video segmentation and recognition
We describe an end-to-end generative approach for the segmentation and recognition of human activities. In this approach, a visual representation based on reduced Fisher Vectors is combined with a structured temporal model for recognition. We show that the statistical properties of Fisher Vectors make them an especially suitable front-end for generative models such as Gaussian mixtures. The system is evaluated for both the recognition of complex activities as well as their parsing into action units. Using a variety of video datasets ranging from human cooking activities to animal behaviors, our experiments demonstrate that the resulting architecture outperforms state-of-the-art approaches for larger datasets, i.e. when sufficient amount of data is available for training structured generative models.
Code (0)
등록된 구현이 없습니다.
Tasks
Video SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videos
This paper focuses on task recognition and action segmentation in weakly-labeled instructional videos, where only the ordered sequence of video-level actions is available during training. We propose a two-stream framewor…
Action SegmentationSegmentationMulti-object Video Generation from Single Frame Layouts
In this paper, we study video synthesis with emphasis on simplifying the generation conditions. Most existing video synthesis models or datasets are designed to address complex motions of a single object, lacking the abi…
Image GenerationObjectVideo GenerationVideo RecognitionEnd-to-End Joint Semantic Segmentation of Actors and Actions in Video
Traditional video understanding tasks include human action recognition and actor/object semantic segmentation. However, the combined task of providing semantic segmentation for different actor classes simultaneously with…
Action RecognitionSegmentationSemantic SegmentationTemporal Action Localization+2REST: REtrieve & Self-Train for generative action recognition
This work is on training a generative action/video recognition model whose output is a free-form action-specific caption describing the video (rather than an action class label). A generative approach has practical advan…
Action RecognitionCaption GenerationContrastive LearningRetrieval+3ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognition
In this paper, we propose a data augmentation method for action recognition using instance segmentation. Although many data augmentation methods have been proposed for image recognition, few of them are tailored for acti…
Action RecognitionData AugmentationInstance SegmentationSegmentation+1