Knowledge Augmented BERT Mutual Network in Multi-turn Spoken Dialogues
Modern spoken language understanding (SLU) systems rely on sophisticated semantic notions revealed in single utterances to detect intents and slots. However, they lack the capability of modeling multi-turn dynamics within a dialogue particularly in long-term slot contexts. Without external knowledge, depending on limited linguistic legitimacy within a word sequence may overlook deep semantic information across dialogue turns. In this paper, we propose to equip a BERT-based joint model with a knowledge attention module to mutually leverage dialogue contexts between two SLU tasks. A gating mechanism is further utilized to filter out irrelevant knowledge triples and to circumvent distracting comprehension. Experimental results in two complicated multi-turn dialogue datasets have demonstrate by mutually modeling two SLU tasks with filtered knowledge and dialogue contexts, our approach has considerable improvements compared with several competitive baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Spoken Language UnderstandingSimilar Papers 제목 키워드 기반
GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning
In this work, we propose a novel framework, Gradient Aligned Mutual Learning BERT (GAML-BERT), for improving the early exiting of BERT. GAML-BERT’s contributions are two-fold. We conduct a set of pilot experiments, which…
Knowledge DistillationInducing Global and Local Knowledge Attention in Multi-turn Dialog Understanding
In multi-turn dialog understanding, semantic frames are constructed by detecting intents and slots within each user utterance. However, recent works lack the capability of modeling multi-turn dynamics within a dialog whe…
Response GenerationSpoken Language UnderstandingInjecting Structured Biomedical Knowledge into Language Models: Continual Pretraining vs. GraphRAG
The injection of domain-specific knowledge is crucial for adapting language models (LMs) to specialized fields such as biomedicine. While most current approaches rely on unstructured text corpora, this study explores two…
Continual PretrainingQuestion AnsweringMG-BERT: Multi-Graph Augmented BERT for Masked Language Modeling
Pre-trained models like Bidirectional Encoder Representations from Transformers (BERT), have recently made a big leap forward in Natural Language Processing (NLP) tasks. However, there are still some shortcomings in the …
Knowledge GraphsLanguage ModelingLanguage ModellingMasked Language Modeling+1Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
In this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (SA-BERT), is proposed in order to make th…
Conversational Response SelectionDisentanglementDomain AdaptationRetrieval