Emotion Understanding in Videos Through Body, Context, and Visual-Semantic Embedding Loss
We present our winning submission to the First International Workshop on Bodily Expressed Emotion Understanding (BEEU) challenge. Based on recent literature on the effect of context/environment on emotion, as well as visual representations with semantic meaning using word embeddings, we extend the framework of Temporal Segment Network to accommodate these. Our method is verified on the validation set of the Body Language Dataset (BoLD) and achieves 0.26235 Emotion Recognition Score on the test set, surpassing the previous best result of 0.2530.
Code (1)
Tasks
Emotion RecognitionWord EmbeddingsSimilar Papers 제목 키워드 기반
MoEmo Vision Transformer: Integrating Cross-Attention and Movement Vectors in 3D Pose Estimation for HRI Emotion Detection
Emotion detection presents challenges to intelligent human-robot interaction (HRI). Foundational deep learning techniques used in emotion detection are limited by information-constrained datasets or models that lack the …
3D Pose EstimationPose EstimationBQA: Body Language Question Answering Dataset for Video Large Language Models
A large part of human communication relies on nonverbal cues such as facial expressions, eye contact, and body language. Unlike language or sign language, such nonverbal communication lacks formal rules, requiring comple…
Question AnsweringBodily expressed emotion understanding through integrating Laban movement analysis
Body movements carry important information about a person's emotions or mental state and are essential in daily communication. Enhancing the ability of machines to understand emotions expressed through body language can …
DiagnosticMapping Emotional Feeling in the Body: A Tripartite Framework for Understanding the Embodied Mind
People often speak of being "moved beyond words" when witnessing awe-inspiring natural beauty, experiencing profound moral elevation, or encountering powerful works of art and music. Such moments evoke intensely embodied…
Enhancing Micro Gesture Recognition for Emotion Understanding via Context-aware Visual-Text Contrastive Learning
Psychological studies have shown that Micro Gestures (MG) are closely linked to human emotions. MG-based emotion understanding has attracted much attention because it allows for emotion understanding through nonverbal bo…
Contrastive LearningGesture RecognitionMicro-gesture Recognition