Cultural-Aware AI Model for Emotion Recognition
Emotion AI is a research domain that aims to understand human emotions from visual or textual data. However, existing methods often ignore the influence of cultural diversity on emotional interpretation. In this paper, we propose a multimodal deep learning model that integrates cultural awareness into emotion recognition. Our model uses images as the primary data source and comments from individuals across different regions as the secondary data source. Our results show that our model achieves robust performance across various scenarios. Our contribution is to introduce a novel fusion approach that bridges cultural gaps and fosters a more nuanced understanding of emotions. Due to the best of our knowledge, few works are using this approach, for Emotion AI, combining different types of data sources and models. We evaluate our model on the ArtELingo dataset, which contains image-comment pairs with Chinese, Arabic, and English annotations. The experimental results in the evaluation phase demonstrate an impressive 80% recognition accuracy for the model that merges image-text features.
Code (1)
Tasks
DiversityEmotion RecognitionmodelMultimodal Deep LearningSimilar Papers 제목 키워드 기반
CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understanding
NLP research has increasingly focused on subjective tasks such as emotion analysis. However, existing emotion benchmarks suffer from two major shortcomings: (1) they largely rely on keyword-based emotion recognition, ove…
BenchmarkingEmotion RecognitionSentiment AnalysisDeciphering Emotions in Children Storybooks: A Comparative Analysis of Multimodal LLMs in Educational Applications
Emotion recognition capabilities in multimodal AI systems are crucial for developing culturally responsive educational technologies, yet remain underexplored for Arabic language contexts where culturally appropriate lear…
Emotion RecognitionCross-cultural Emotion Classification: the Effect of Emotional Intensity and Acoustic Features
Cross-cultural emotion recognition is attracting increasing research attention; robustness to such differences in emotional expression is important for speech modality emotion recognition. In this work we quantify the ac…
Cultural Vocal Bursts Intensity PredictionEmotion ClassificationEmotion RecognitionFeature ImportanceTowards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures
In our multicultural world, affect-aware AI systems that support humans need the ability to perceive affect across variations in emotion expression patterns across cultures. These systems must perform well in cultural co…
Causal DiscoveryCultural Vocal Bursts Intensity Predictionfeature selectionBeyond Universality: The GCC-FER Dataset and Culture-Aware Adaptation for Dynamic Facial Expression Recognition
Dynamic Facial Expression Recognition (DFER) is a key enabling technology in affective computing, human-computer interaction, and intelligent multimedia systems. Despite the significant influence of cultural nuances on F…
Facial Expression Recognition