Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis
Building Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors is an essential issue for various voice-enabled virtual assistants. Considering that most ASR errors are caused by phonetic confusion between similar-sounding expressions, intuitively, leveraging the phoneme sequence of speech can complement ASR hypothesis and enhance the robustness of SLU. This paper proposes a novel model with Cross Attention for SLU (denoted as CASLU). The cross attention block is devised to catch the fine-grained interactions between phoneme and word embeddings in order to make the joint representations catch the phonetic and semantic features of input simultaneously and for overcoming the ASR errors in downstream natural language understanding (NLU) tasks. Extensive experiments are conducted on three datasets, showing the effectiveness and competitiveness of our approach. Additionally, We also validate the universality of CASLU and prove its complementarity when combining with other robust SLU techniques.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Natural Language Understandingspeech-recognitionSpeech RecognitionSpoken Language UnderstandingWord EmbeddingsSimilar Papers 제목 키워드 기반
Transferring SLU Models in Novel Domains
Spoken language understanding (SLU) is a critical component in building dialogue systems. When building models for novel natural language domains, a major challenge is the lack of data in the new domains, no matter wheth…
Intent RecognitionMeta-Learningslot-fillingSlot Filling+2Finstreder: Simple and fast Spoken Language Understanding with Finite State Transducers using modern Speech-to-Text models
In Spoken Language Understanding (SLU) the task is to extract important information from audio commands, like the intent of what a user wants the system to do and special entities like locations or numbers. This paper pr…
Intent ClassificationSlot FillingSpeech-to-TextSpoken Language UnderstandingA Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding
Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data, which has gained increasing attention. In this work, we investigate the understanding ability of ChatGPT for z…
Dialogue State TrackingDialogue Understandingslot-fillingSlot Filling+1On Building Spoken Language Understanding Systems for Low Resourced Languages
Spoken dialog systems are slowly becoming and integral part of the human experience due to their various advantages over textual interfaces. Spoken language understanding (SLU) systems are fundamental building blocks of …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)intent-classificationIntent Classification+3Improving End-to-End Models for Set Prediction in Spoken Language Understanding
The goal of spoken language understanding (SLU) systems is to determine the meaning of the input speech signal, unlike speech recognition which aims to produce verbatim transcripts. Advances in end-to-end (E2E) speech mo…
Data AugmentationDecoderspeech-recognitionSpeech Recognition+1