Context-aware Biaffine Localizing Network for Temporal Sentence Grounding
This paper addresses the problem of temporal sentence grounding (TSG), which aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. Previous works either compare pre-defined candidate segments with the query and select the best one by ranking, or directly regress the boundary timestamps of the target segment. In this paper, we propose a novel localization framework that scores all pairs of start and end indices within the video simultaneously with a biaffine mechanism. In particular, we present a Context-aware Biaffine Localizing Network (CBLN) which incorporates both local and global contexts into features of each start/end position for biaffine-based localization. The local contexts from the adjacent frames help distinguish the visually similar appearance, and the global contexts from the entire video contribute to reasoning the temporal relation. Besides, we also develop a multi-modal self-attention module to provide fine-grained query-guided video representation for this biaffine strategy. Extensive experiments show that our CBLN significantly outperforms state-of-the-arts on three public datasets (ActivityNet Captions, TACoS, and Charades-STA), demonstrating the effectiveness of the proposed localization framework.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceTemporal Sentence GroundingSimilar Papers 제목 키워드 기반
Entity-Aware Biaffine Attention Model for Improved Constituent Parsing with Reduced Entity Violations
Constituency parsing involves analyzing a sentence by breaking it into sub-phrases, or constituents. While many deep neural models have achieved state-of-the-art performance in this task, they often overlook the entity-v…
Constituency ParsingSentenceSentiment AnalysisLocalizing Moments in Video with Temporal Language
Localizing moments in a longer video via natural language queries is a new, challenging task at the intersection of language and video understanding. Though moment localization with natural language is similar to other l…
Natural Language QueriesRetrievalVideo UnderstandingA Full End-to-End Semantic Role Labeler, Syntax-agnostic Over Syntax-aware?
Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling. Previous studies usually formulate the entire SRL problem…
DecoderSemantic Role LabelingSentenceA Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware?
Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling. Previous studies usually formulate the entire SRL problem…
DecoderMachine TranslationQuestion AnsweringSemantic Parsing+2JBNU at MRP 2019: Multi-level Biaffine Attention for Semantic Dependency Parsing
This paper describes Jeonbuk National University (JBNU){'}s system for the 2019 shared task on Cross-Framework Meaning Representation Parsing (MRP 2019) at the Conference on Computational Natural Language Learning. Of th…
DecoderDependency ParsingSemantic Dependency ParsingSentence