Improving End-to-End Models for Set Prediction in Spoken Language Understanding
The goal of spoken language understanding (SLU) systems is to determine the meaning of the input speech signal, unlike speech recognition which aims to produce verbatim transcripts. Advances in end-to-end (E2E) speech modeling have made it possible to train solely on semantic entities, which are far cheaper to collect than verbatim transcripts. We focus on this set prediction problem, where entity order is unspecified. Using two classes of E2E models, RNN transducers and attention based encoder-decoders, we show that these models work best when the training entity sequence is arranged in spoken order. To improve E2E SLU models when entity spoken order is unknown, we propose a novel data augmentation technique along with an implicit attention based alignment method to infer the spoken order. F1 scores significantly increased by more than 11% for RNN-T and about 2% for attention based encoder-decoder SLU models, outperforming previously reported results.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationDecoderspeech-recognitionSpeech RecognitionSpoken Language UnderstandingSimilar Papers 제목 키워드 기반
What BERT Based Language Models Learn in Spoken Transcripts: An Empirical Study
Language Models (LMs) have been ubiquitously leveraged in various tasks including spoken language understanding (SLU). Spoken language requires careful understanding of speaker interactions, dialog states and speech indu…
Spoken Language UnderstandingWhat BERT Based Language Model Learns in Spoken Transcripts: An Empirical Study
Language Models (LMs) have been ubiquitously leveraged in various tasks including spoken language understanding (SLU). Spoken language requires careful understanding of speaker interactions, dialog states and speech indu…
Language ModelingLanguage ModellingSpoken Language UnderstandingBidirectional Representations for Low Resource Spoken Language Understanding
Most spoken language understanding systems use a pipeline approach composed of an automatic speech recognition interface and a natural language understanding module. This approach forces hard decisions when converting co…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModellingNatural Language Understanding+3I$^2$KD-SLU: An Intra-Inter Knowledge Distillation Framework for Zero-Shot Cross-Lingual Spoken Language Understanding
Spoken language understanding (SLU) typically includes two subtasks: intent detection and slot filling. Currently, it has achieved great success in high-resource languages, but it still remains challenging in low-resourc…
Intent DetectionKnowledge Distillationslot-fillingSlot Filling+1A Hierarchical Decoding Model For Spoken Language Understanding From Unaligned Data
Spoken language understanding (SLU) systems can be trained on two types of labelled data: aligned or unaligned. Unaligned data do not require word by word annotation and is easier to be obtained. In the paper, we focus o…
Spoken Language Understanding