MELD-ST: An Emotion-aware Speech Translation Dataset
Emotion plays a crucial role in human conversation. This paper underscores the significance of considering emotion in speech translation. We present the MELD-ST dataset for the emotion-aware speech translation task, comprising English-to-Japanese and English-to-German language pairs. Each language pair includes about 10,000 utterances annotated with emotion labels from the MELD dataset. Baseline experiments using the SeamlessM4T model on the dataset indicate that fine-tuning with emotion labels can enhance translation performance in some settings, highlighting the need for further research in emotion-aware speech translation systems.
Code (0)
등록된 구현이 없습니다.
Tasks
TranslationSimilar Papers 제목 키워드 기반
Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge
The task of emotion recognition in conversations (ERC) benefits from the availability of multiple modalities, as provided, for example, in the video-based Multimodal EmotionLines Dataset (MELD). However, only a few resea…
Active Speaker DetectionAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)Emotion Recognition+2A Change of Heart: Improving Speech Emotion Recognition through Speech-to-Text Modality Conversion
Speech Emotion Recognition (SER) is a challenging task. In this paper, we introduce a modality conversion concept aimed at enhancing emotion recognition performance on the MELD dataset. We assess our approach through two…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Emotion RecognitionSpeech Emotion Recognition+3Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs
Audio Large Language Models (AudioLLMs) have achieved strong results in semantic tasks like speech recognition and translation, but remain limited in modeling paralinguistic cues such as emotion. Existing approaches ofte…
Emotion Recognitionspeech-recognitionSpeech RecognitionM-MELD: A Multilingual Multi-Party Dataset for Emotion Recognition in Conversations
Expression of emotions is a crucial part of daily human communication. Emotion recognition in conversations (ERC) is an emerging field of study, where the primary task is to identify the emotion behind each utterance in …
Emotion RecognitionMELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
Emotion recognition in conversations is a challenging task that has recently gained popularity due to its potential applications. Until now, however, a large-scale multimodal multi-party emotional conversational database…
Dialogue GenerationEmotion RecognitionEmotion Recognition in Conversation