Emotional Reaction Intensity Estimation Based on Multimodal Data

2023-03-16 · Shangfei Wang, Jiaqiang Wu, Feiyi Zheng, Xin Li, XueWei Li, Suwen Wang, Yi Wu, Yanan Chang, Xiangyu Miao

This paper introduces our method for the Emotional Reaction Intensity (ERI) Estimation Challenge, in CVPR 2023: 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). Based on the multimodal data provided by the originazers, we extract acoustic and visual features with different pretrained models. The multimodal features are mixed together by Transformer Encoders with cross-modal attention mechnism. In this paper, 1. better features are extracted with the SOTA pretrained models. 2. Compared with the baseline, we improve the Pearson's Correlations Coefficient a lot. 3. We process the data with some special skills to enhance performance ability of our model.

📄 PDF Abstract BibTeX arXiv:2303.09167

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Position-Wise Feed-Forward Layer 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Residual Connection 설명 없음

Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers

2023-03-16 · Jia Li, Yin Chen, Xuesong Zhang, Jiantao Nie 외

In this paper, we present our advanced solutions to the two sub-challenges of Affective Behavior Analysis in the wild (ABAW) 2023: the Emotional Reaction Intensity (ERI) Estimation Challenge and Expression (Expr) Classif…

Classification

A Dual Branch Network for Emotional Reaction Intensity Estimation

2023-03-16 · Jun Yu, Jichao Zhu, Wangyuan Zhu, Zhongpeng Cai 외

Emotional Reaction Intensity(ERI) estimation is an important task in multimodal scenarios, and has fundamental applications in medicine, safe driving and other fields. In this paper, we propose a solution to the ERI chal…

regression

Computer Vision Estimation of Emotion Reaction Intensity in the Wild

2023-03-19 · Yang Qian, Ali Kargarandehkordi, Onur Cezmi Mutlu, Saimourya Surabhi 외

Emotions play an essential role in human communication. Developing computer vision models for automatic recognition of emotion expression can aid in a variety of domains, including robotics, digital behavioral healthcare…

Human Reaction Intensity Estimation with Ensemble of Multi-task Networks

2023-03-16 · JiYeon Oh, Daun Kim, Jae-Yeop Jeong, Yeong-Gi Hong 외

Facial expression in-the-wild is essential for various interactive computing domains. Especially, "Emotional Reaction Intensity" (ERI) is an important topic in the facial expression recognition task. In this paper, we pr…

Facial Expression Recognition

ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges

2023-03-02 · Dimitrios Kollias, Panagiotis Tzirakis, Alice Baird, Alan Cowen 외

The fifth Affective Behavior Analysis in-the-wild (ABAW) Competition is part of the respective ABAW Workshop which will be held in conjunction with IEEE Computer Vision and Pattern Recognition Conference (CVPR), 2023. Th…

Action Unit Detection Arousal Estimation