Topic Classification on Spoken Documents Using Deep Acoustic and Linguistic Features
Topic classification systems on spoken documents usually consist of two modules: an automatic speech recognition (ASR) module to convert speech into text and a text topic classification (TTC) module to predict the topic class from the decoded text. In this paper, instead of using the ASR transcripts, the fusion of deep acoustic and linguistic features is used for topic classification on spoken documents. More specifically, a conventional CTC-based acoustic model (AM) using phonemes as output units is first trained, and the outputs of the layer before the linear phoneme classifier in the trained AM are used as the deep acoustic features of spoken documents. Furthermore, these deep acoustic features are fed to a phoneme-to-word (P2W) module to obtain deep linguistic features. Finally, a local multi-head attention module is proposed to fuse these two types of deep features for topic classification. Experiments conducted on a subset selected from Switchboard corpus show that our proposed framework outperforms the conventional ASR+TTC systems and achieves a 3.13% improvement in ACC.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Classificationspeech-recognitionSpeech RecognitionTopic ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification
Intent classification is a task in spoken language understanding. An intent classification system is usually implemented as a pipeline process, with a speech recognition module followed by text processing that classifies…
ClassificationGeneral Classificationintent-classificationIntent Classification+6Audio-Linguistic Embeddings for Spoken Sentences
We propose spoken sentence embeddings which capture both acoustic and linguistic content. While existing works operate at the character, phoneme, or word level, our method learns long-term dependencies by modeling speech…
DecoderEmotion RecognitionSentenceSentence Embeddings+3Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling
Textless spoken language models (SLMs) are generative models of speech that do not rely on text supervision. Most textless SLMs learn to predict the next semantic token, a discrete representation of linguistic content, a…
100,000 Podcasts: A Spoken English Document Corpus
Podcasts are a large and growing repository of spoken audio. As an audio format, podcasts are more varied in style and production type than broadcast news, contain more genres than typically studied in video data, and ar…
3D Facial Landmark LocalizationAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)Facial Expression Recognition (FER)+5Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversations
Due to the lack of publicly available resources, conversation summarization has received far less attention than text summarization. As the purpose of conversations is to exchange information between at least two interlo…
Conversation SummarizationExtractive SummarizationSentenceText Summarization