Double Multi-Head Attention Multimodal System for Odyssey 2024 Speech Emotion Recognition Challenge

2024-06-15 · Federico Costa, Miquel India, Javier Hernando

As computer-based applications are becoming more integrated into our daily lives, the importance of Speech Emotion Recognition (SER) has increased significantly. Promoting research with innovative approaches in SER, the Odyssey 2024 Speech Emotion Recognition Challenge was organized as part of the Odyssey 2024 Speaker and Language Recognition Workshop. In this paper we describe the Double Multi-Head Attention Multimodal System developed for this challenge. Pre-trained self-supervised models were used to extract informative acoustic and text features. An early fusion strategy was adopted, where a Multi-Head Attention layer transforms these mixed features into complementary contextualized representations. A second attention mechanism is then applied to pool these representations into an utterance-level vector. Our proposed system achieved the third position in the categorical task ranking with a 34.41% Macro-F1 score, where 31 teams participated in total.

📄 PDF Abstract BibTeX arXiv:2406.10598

Code (0)

등록된 구현이 없습니다.

Tasks

Emotion RecognitionPositionSpeech Emotion Recognition

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs

2026-02-05 · Wentao Ni, Kangqi Zhang, Zhongming Yu, Oren Nelson 외 arxiv

As long-context inference becomes central to large language models (LLMs), attention over growing key-value caches emerges as a dominant decoding bottleneck, motivating sparse attention for scalable inference. Fixed-budg…

Conditional Diffusion Model-Driven Generative Channels for Double RIS-Aided Wireless Systems

2025-06-15 · Yiyang Ni, Qi Zhang, Guangji Chen, Yan Cai 외

With the development of the upcoming sixth-generation networks (6G), reconfigurable intelligent surfaces (RISs) have gained significant attention due to its ability of reconfiguring wireless channels via smart reflection…

MATT: Multimodal Attention Level Estimation for e-learning Platforms

2023-01-22 · Roberto Daza, Luis F. Gomez, Aythami Morales, Julian Fierrez 외

This work presents a new multimodal system for remote attention level estimation based on multimodal face analysis. Our multimodal approach uses different parameters and signals obtained from the behavior and physiologic…

Facial Landmark Detection Head Pose Estimation Pose Estimation

CADMR: Cross-Attention and Disentangled Learning for Multimodal Recommender Systems

2024-12-03 · Yasser Khalafaoui, Martino Lovisetto, Basarab Matei, Nistor Grozavu

The increasing availability and diversity of multimodal data in recommender systems offer new avenues for enhancing recommendation accuracy and user satisfaction. However, these systems must contend with high-dimensional…

Diversity Recommendation Systems

Double Attention-based Multimodal Neural Machine Translation with Semantic Image Regions

2020-11-03 · EAMT 2020 11 · YuTing Zhao, Mamoru Komachi, Tomoyuki Kajiwara, Chenhui Chu

Existing studies on multimodal neural machine translation (MNMT) have mainly focused on the effect of combining visual and textual modalities to improve translations. However, it has been suggested that the visual modali…

Machine Translation Translation