Using Emotion Embeddings to Transfer Knowledge Between Emotions, Languages, and Annotation Formats
The need for emotional inference from text continues to diversify as more and more disciplines integrate emotions into their theories and applications. These needs include inferring different emotion types, handling multiple languages, and different annotation formats. A shared model between different configurations would enable the sharing of knowledge and a decrease in training costs, and would simplify the process of deploying emotion recognition models in novel environments. In this work, we study how we can build a single model that can transition between these different configurations by leveraging multilingual models and Demux, a transformer-based model whose input includes the emotions of interest, enabling us to dynamically change the emotions predicted by the model. Demux also produces emotion embeddings, and performing operations on them allows us to transition to clusters of emotions by pooling the embeddings of each cluster. We show that Demux can simultaneously transfer knowledge in a zero-shot manner to a new language, to a novel annotation format and to unseen emotions. Code is available at https://github.com/gchochla/Demux-MEmo .
Code (1)
Tasks
Emotion RecognitionSimilar Papers 제목 키워드 기반
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition
Despite the recent achievements made in the multi-modal emotion recognition task, two problems still exist and have not been well investigated: 1) the relationship between different emotion categories are not utilized, w…
Emotion RecognitionMultimodal Emotion RecognitionWord EmbeddingsCross-Modal Emotion Transfer for Emotion Editing in Talking Face Video
Talking face generation has gained significant attention as a core application of generative models. To enhance the expressiveness and realism of synthesized videos, emotion editing in talking face video plays a crucial …
Talking Face GenerationAffectEcho: Speaker Independent and Language-Agnostic Emotion and Affect Transfer for Speech Synthesis
Affect is an emotional characteristic encompassing valence, arousal, and intensity, and is a crucial attribute for enabling authentic conversations. While existing text-to-speech (TTS) and speech-to-speech systems rely o…
AttributeSpeech Synthesistext-to-speechText to SpeechA Protocol for Emotions
We tend to consider emotions a manifestation of our innermost nature of human beings. Emotions characterize our lives in many ways and they chaperon every rational activity we carry out. Despite their pervasiveness, ther…
Cross Domain Emotion Recognition using Few Shot Knowledge Transfer
Emotion recognition from text is a challenging task due to diverse emotion taxonomies, lack of reliable labeled data in different domains, and highly subjective annotation standards. Few-shot and zero-shot techniques can…
Emotion RecognitionTransfer Learning