Semantic Frames and Visual Scenes: Learning Semantic Role Inventories from Image and Video Descriptions
Frame-semantic parsing and semantic role labelling, that aim to automatically assign semantic roles to arguments of verbs in a sentence, have become an active strand of research in NLP. However, to date these methods have relied on a predefined inventory of semantic roles. In this paper, we present a method to automatically learn argument role inventories for verbs from large corpora of text, images and videos. We evaluate the method against manually constructed role inventories in FrameNet and show that the visual model outperforms the language-only model and operates with a high precision.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringSemantic ParsingSentenceSimilar Papers 제목 키워드 기반
Semantic Flow: Learning Semantic Field of Dynamic Scenes from Monocular Videos
In this work, we pioneer Semantic Flow, a neural semantic representation of dynamic scenes from monocular videos. In contrast to previous NeRF methods that reconstruct dynamic scenes from the colors and volume densities …
NeRFConstructing Web-Accessible Semantic Role Labels and Frames for Japanese as Additions to the NPCMJ Parsed Corpus
As part of constructing the NINJAL Parsed Corpus of Modern Japanese (NPCMJ), a web-accessible language resource, we are adding frame information for predicates, together with two types of semantic role labels that mark t…
AMR ParsingSemantic ParsingImproving Chinese Semantic Role Labeling using High-quality Surface and Deep Case Frames
This paper presents a method for applying automatically acquired knowledge to semantic role labeling (SRL). We use a large amount of automatically extracted knowledge to improve the performance of SRL. We present two var…
Chinese Semantic Role LabelingDependency ParsingMachine TranslationManagement+2Semantic-Aware Dynamic Parameter for Video Inpainting Transformer
Recent learning-based video inpainting approaches have achieved considerable progress. However, they still cannot fully utilize semantic information within the video frames and predict improper scene layout, failing …
Mixture-of-ExpertsVideo InpaintingFeasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model
Frame-wise semantic segmentation of indoor lidar scans is a fundamental step toward higher-level 3D scene understanding and mapping applications. However, acquiring frame-wise ground truth for training deep learning mode…
LIDAR Semantic SegmentationScene UnderstandingAutonomous Driving