Unsupervised Semantic Parsing of Video Collections
Human communication typically has an underlying structure. This is reflected in the fact that in many user generated videos, a starting point, ending, and certain objective steps between these two can be identified. In this paper, we propose a method for parsing a video into such semantic steps in an unsupervised way. The proposed method is capable of providing a semantic "storyline" of the video composed of its objective steps. We accomplish this using both visual and language cues in a joint generative model. The proposed method can also provide a textual description for each of the identified semantic steps and video segments. We evaluate this method on a large number of complex YouTube videos and show results of unprecedented quality for this intricate and impactful problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic ParsingUnsupervised semantic parsingSimilar Papers 제목 키워드 기반
Unsupervised Semantic Action Discovery from Video Collections
Human communication takes many forms, including speech, text and instructional videos. It typically has an underlying structure, with a starting point, ending, and certain objective steps between them. In this paper, we …
UBoCo : Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detection
Generic Event Boundary Detection (GEBD) is a newly suggested video understanding task that aims to find one level deeper semantic boundaries of events. Bridging the gap between natural human perception and video understa…
Boundary DetectionContrastive LearningGeneric Event Boundary Detectionvalid+1UBoCo: Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detection
Generic Event Boundary Detection (GEBD) is a newly suggested video understanding task that aims to find one level deeper semantic boundaries of events. Bridging the gap between natural human perception and video unde…
Boundary DetectionContrastive LearningGeneric Event Boundary Detectionvalid+1Semantic Parsing of Colonoscopy Videos with Multi-Label Temporal Networks
Following the successful debut of polyp detection and characterization, more advanced automation tools are being developed for colonoscopy. The new automation tasks, such as quality metrics or report generation, require …
Semantic ParsingFrom Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decoding
Semantic parsing is challenging due to the structure gap and the semantic gap between utterances and logical forms. In this paper, we propose an unsupervised semantic parsing method - Synchronous Semantic Decoding (SSD),…
FormSemantic ParsingUnsupervised semantic parsing