Coreference by Appearance: Visually Grounded Event Coreference Resolution
Event coreference resolution is critical to understand events in the growing number of online news with multiple modalities including text, video, speech, etc. However, the events and entities depicting in different modalities may not be perfectly aligned and can be difficult to annotate, which makes the task especially challenging with little supervision available. To address the above issues, we propose a supervised model based on attention mechanism and an unsupervised model based on statistical machine translation, capable of learning the relative importance of modalities for event coreference resolution. Experiments on a video multimedia event dataset show that our multimodal models outperform text-only systems in event coreference resolution tasks. A careful analysis reveals that the performance gain of the multimodal model especially under unsupervised settings comes from better learning of visually salient events.
Code (0)
등록된 구현이 없습니다.
Tasks
coreference-resolutionCoreference ResolutionEvent Coreference ResolutionMachine TranslationTranslationSimilar Papers 제목 키워드 기반
Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context. In addition to the challenges found in visual question answering (VQA), which can be seen as one-round dialog, …
Common Sense Reasoningcoreference-resolutionCoreference ResolutionVisual Dialog+3SinoCoreferencer: An End-to-End Chinese Event Coreference Resolver
Compared to entity coreference resolution, there is a relatively small amount of work on event coreference resolution. Much work on event coreference was done for English. In fact, to our knowledge, there are no publicly…
coreference-resolutionCoreference ResolutionEntity Extraction using GANEntity Typing+3Constrained Multi-Task Learning for Event Coreference Resolution
We propose a neural event coreference model in which event coreference is jointly trained with five tasks: trigger detection, entity coreference, anaphoricity determination, realis detection, and argument extraction. To …
coreference-resolutionCoreference ResolutionEvent Coreference ResolutionMulti-Task LearningImproving Event Coreference Resolution by Modeling Correlations between Event Coreference Chains and Document Topic Structures
This paper proposes a novel approach for event coreference resolution that models correlations between event coreference chains and document topical structures through an Integer Linear Programming formulation. We explic…
coreference-resolutionCoreference ResolutionEvent Coreference ResolutionQuestion AnsweringEnd-to-End Neural Event Coreference Resolution
Traditional event coreference systems usually rely on pipeline framework and hand-crafted features, which often face error propagation problem and have poor generalization ability. In this paper, we propose an End-to-End…
coreference-resolutionCoreference ResolutionEvent Coreference ResolutionEvent Detection