A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding
Spoken Language Understanding (SLU), which typically involves intent determination and slot filling, is a core component of spoken dialogue systems. Joint learning has shown to be effective in SLU given that slot tags and intents are supposed to share knowledge with each other. However, most existing joint learning methods only consider joint learning by sharing parameters on surface level rather than semantic level. In this work, we propose a novel self-attentive model with gate mechanism to fully utilize the semantic correlation between slot and intent. Our model first obtains intent-augmented embeddings based on neural network with self-attention mechanism. And then the intent semantic representation is utilized as the gate for labelling slot tags. The objectives of both tasks are optimized simultaneously via joint learning in an end-to-end way. We conduct experiment on popular benchmark ATIS. The results show that our model achieves state-of-the-art and outperforms other popular methods by a large margin in terms of both intent detection error rate and slot filling F1-score. This paper gives a new perspective for research on SLU.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech Recognition (ASR)Intent DetectionLanguage ModelingLanguage Modellingslot-fillingSlot FillingSpeech RecognitionSpoken Dialogue SystemsSpoken Language UnderstandingSimilar Papers 제목 키워드 기반
Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding
Abstractive community detection is an important spoken language understanding task, whose goal is to group utterances in a conversation according to whether they can be jointly summarized by a common abstractive sentence…
ClusteringCommunity DetectionSentenceSpoken Language Understanding+1Language ID Prediction from Speech Using Self-Attentive Pooling
This memo describes NTR-TSU submission for SIGTYP 2021 Shared Task on predicting language IDs from speech. Spoken Language Identification (LID) is an important step in a multilingual Automated Speech Recognition (ASR) sy…
Language Identificationspeech-recognitionSpeech RecognitionSpoken language identificationLanguage ID Prediction from Speech Using Self-Attentive Pooling and 1D-Convolutions
This memo describes NTR-TSU submission for SIGTYP 2021 Shared Task on predicting language IDs from speech. Spoken Language Identification (LID) is an important step in a multilingual Automated Speech Recognition (ASR) sy…
Language Identificationspeech-recognitionSpeech RecognitionSpoken language identificationLow-Resource Spoken Language Identification Using Self-Attentive Pooling and Deep 1D Time-Channel Separable Convolutions
This memo describes NTR/TSU winning submission for Low Resource ASR challenge at Dialog2021 conference, language identification track. Spoken Language Identification (LID) is an important step in a multilingual Automated…
Language Identificationspeech-recognitionSpeech RecognitionSpoken language identificationOn the Robustness of Self-Attentive Models
This work examines the robustness of self-attentive neural networks against adversarial input perturbations. Specifically, we investigate the attention and feature extraction mechanisms of state-of-the-art recurrent neur…
Machine TranslationSentiment AnalysisTranslation