Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech
Cross-modality between vision and language is a key component for effective and efficient communication, and human language processing mechanism successfully integrates information from various modalities to extract the intended meaning. However, incomplete linguistic input, i.e. due to a noisy environment, is one of the challenges for a successful communication. In that case, an incompleteness in one channel can be compensated by information from another one. In this paper, by conducting visual-world paradigm, we investigated the dynamics between syntactically possible gap fillers and the visual arrangements in incomplete German sentences and their effect on overall sentence interpretation.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language UnderstandingSentenceSimilar Papers 제목 키워드 기반
Enhancing Cross-lingual Natural Language Inference by Prompt-learning from Cross-lingual Templates
Cross-lingual natural language inference (XNLI) is a fundamental task in cross-lingual natural language understanding. Recently this task is commonly addressed by pre-trained cross-lingual language models. Existing metho…
Cross-Lingual Natural Language InferenceCross-Lingual TransferLanguage ModelingLanguage Modelling+4Unified BERT for Few-shot Natural Language Understanding
Even as pre-trained language models share a semantic encoder, natural language understanding suffers from a diversity of output schemas. In this paper, we propose UBERT, a unified bidirectional language understanding mod…
DiversityNatural Language Understanding$C^3$: Confidence Calibration Model Cascade for Inference-Efficient Cross-Lingual Natural Language Understanding
Cross-lingual natural language understanding (NLU) is a critical task in natural language processing (NLP). Recent advancements have seen multilingual pre-trained language models (mPLMs) significantly enhance the perform…
Natural Language UnderstandingEnhancing Reinforcement Learning with Label-Sensitive Reward for Natural Language Understanding
Recent strides in large language models (LLMs) have yielded remarkable performance, leveraging reinforcement learning from human feedback (RLHF) to significantly enhance generation and alignment capabilities. However, RL…
Natural Language Understandingreinforcement-learningReinforcement LearningWhen Graph Data Meets Multimodal: A New Paradigm for Graph Understanding and Reasoning
Graph data is ubiquitous in the physical world, and it has always been a challenge to efficiently model graph structures using a unified paradigm for the understanding and reasoning on various graphs. Moreover, in the er…
Optical Character Recognition (OCR)