Exploring Textual and Speech information in Dialogue Act Classification with Speaker Domain Adaptation
In spite of the recent success of Dialogue Act (DA) classification, the majority of prior works focus on text-based classification with oracle transcriptions, i.e. human transcriptions, instead of Automatic Speech Recognition (ASR)'s transcriptions. In spoken dialog systems, however, the agent would only have access to noisy ASR transcriptions, which may further suffer performance degradation due to domain shift. In this paper, we explore the effectiveness of using both acoustic and textual signals, either oracle or ASR transcriptions, and investigate speaker domain adaptation for DA classification. Our multimodal model proves to be superior to the unimodal models, particularly when the oracle transcriptions are not available. We also propose an effective method for speaker domain adaptation, which achieves competitive results.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)ClassificationDialogue Act ClassificationDomain AdaptationGeneral Classificationspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking
Dialogue state tracking plays a crucial role in extracting information in task-oriented dialogue systems. However, preceding research are limited to textual modalities, primarily due to the shortage of authentic human au…
Dialogue State TrackingTask-Oriented Dialogue SystemsContextual Speech Extraction: Leveraging Textual History as an Implicit Cue for Target Speech Extraction
In this paper, we investigate a novel approach for Target Speech Extraction (TSE), which relies solely on textual context to extract the target speech. We refer to this task as Contextual Speech Extraction (CSE). Unlike …
Speech ExtractionExploring Speech Pattern Disorders in Autism using Machine Learning
Diagnosing autism spectrum disorder (ASD) by identifying abnormal speech patterns from examiner-patient dialogues presents significant challenges due to the subtle and diverse manifestations of speech-related symptoms in…
DiagnosticregressionRhythmInterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations
While recently developed NLP explainability methods let us open the black box in various ways (Madsen et al., 2022), a missing ingredient in this endeavor is an interactive tool offering a conversational interface. Such …
Dialogue Act ClassificationHate Speech DetectionQuestion AnsweringSLIDE: Integrating Speech Language Model with LLM for Spontaneous Spoken Dialogue Generation
Recently, ``textless" speech language models (SLMs) based on speech units have made huge progress in generating naturalistic speech, including non-verbal vocalizations. However, the generated speech samples often lack se…
Dialogue GenerationLanguage ModelingLanguage Modelling