Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification
Intent classification is a task in spoken language understanding. An intent classification system is usually implemented as a pipeline process, with a speech recognition module followed by text processing that classifies the intents. There are also studies of end-to-end system that takes acoustic features as input and classifies the intents directly. Such systems don't take advantage of relevant linguistic information, and suffer from limited training data. In this work, we propose a novel intent classification framework that employs acoustic features extracted from a pretrained speech recognition system and linguistic features learned from a pretrained language model. We use knowledge distillation technique to map the acoustic embeddings towards linguistic embeddings. We perform fusion of both acoustic and linguistic embeddings through cross-attention approach to classify intents. With the proposed method, we achieve 90.86% and 99.07% accuracy on ATIS and Fluent speech corpus, respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral Classificationintent-classificationIntent ClassificationKnowledge DistillationLanguage ModelingLanguage Modellingspeech-recognitionSpeech RecognitionSpoken Language UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Demographic Attributes Prediction from Speech Using WavLM Embeddings
This paper introduces a general classifier based on WavLM features, to infer demographic characteristics, such as age, gender, native language, education, and country, from speech. Demographic feature prediction plays a …
DiversityGender ClassificationPredictionListening Between the Lines: Joint Learning of ASR Embeddings and LLM-Augmented Linguistics for Dementia Detection
Early detection of dementia through speech analysis offers a non-invasive screening alternative, but capturing both acoustic and linguistic biomarkers remains challenging. We propose a multimodal framework leveraging Whi…
Speech RecognitionParalinguistics-Enhanced Large Language Modeling of Spoken Dialogue
Large Language Models (LLMs) have demonstrated superior abilities in tasks such as chatting, reasoning, and question-answering. However, standard LLMs may ignore crucial paralinguistic information, such as sentiment, emo…
AttributeLanguage ModelingLanguage ModellingQuestion Answering+3Audio-Linguistic Embeddings for Spoken Sentences
We propose spoken sentence embeddings which capture both acoustic and linguistic content. While existing works operate at the character, phoneme, or word level, our method learns long-term dependencies by modeling speech…
DecoderEmotion RecognitionSentenceSentence Embeddings+3CTA-RNN: Channel and Temporal-wise Attention RNN Leveraging Pre-trained ASR Embeddings for Speech Emotion Recognition
Previous research has looked into ways to improve speech emotion recognition (SER) by utilizing both acoustic and linguistic cues of speech. However, the potential association between state-of-the-art ASR models and the …
Cross-corpusEmotion RecognitionSpeech Emotion Recognition