Learning Latent Semantic Annotations for Grounding Natural Language to Structured Data
Previous work on grounded language learning did not fully capture the semantics underlying the correspondences between structured world state representations and texts, especially those between numerical values and lexical terms. In this paper, we attempt at learning explicit latent semantic annotations from paired structured tables and texts, establishing correspondences between various types of values and texts. We model the joint probability of data fields, texts, phrasal spans, and latent annotations with an adapted semi-hidden Markov model, and impose a soft statistical constraint to further improve the performance. As a by-product, we leverage the induced annotations to extract templates for language generation. Experimental results suggest the feasibility of the setting in this study, as well as the effectiveness of our proposed framework.
Code (1)
Tasks
Grounded language learningLanguage AcquisitionText GenerationSimilar Papers 제목 키워드 기반
Panoptic Narrative Grounding
This paper proposes Panoptic Narrative Grounding, a spatially fine and general formulation of the natural language visual grounding problem. We establish an experimental framework for the study of this new task, includin…
Natural Language Visual GroundingPanoptic SegmentationVisual GroundingPanoptic Narrative Grounding
This paper proposes Panoptic Narrative Grounding, a spatially fine and general formulation of the natural language visual grounding problem. We establish an experimental framework for the study of this new task, incl…
Natural Language Visual GroundingPanoptic SegmentationVisual GroundingWeakly-Supervised 3D Visual Grounding based on Visual Linguistic Alignment
Learning to ground natural language queries to target objects or regions in 3D point clouds is quite essential for 3D scene understanding. Nevertheless, existing 3D visual grounding approaches require a substantial numbe…
3D visual groundingNatural Language QueriesScene UnderstandingVisual GroundingThink Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehicles
Interpreting natural-language commands to localize target objects is critical for autonomous driving (AD). Existing visual grounding (VG) methods for autonomous vehicles (AVs) typically struggle with ambiguous, context-d…
Autonomous VehiclesAutonomous DrivingVisual GroundingFine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding
Temporal language grounding (TLG) aims to localize a video segment in an untrimmed video based on a natural language description. To alleviate the expensive cost of manual annotations for temporal boundary labels, we are…
cross-modal alignmentSentence