Reading Between the Frames: Multi-Modal Depression Detection in Videos from Non-Verbal Cues
Depression, a prominent contributor to global disability, affects a substantial portion of the population. Efforts to detect depression from social media texts have been prevalent, yet only a few works explored depression detection from user-generated video content. In this work, we address this research gap by proposing a simple and flexible multi-modal temporal model capable of discerning non-verbal depression cues from diverse modalities in noisy, real-world videos. We show that, for in-the-wild videos, using additional high-level non-verbal cues is crucial to achieving good performance, and we extracted and processed audio speech embeddings, face emotion embeddings, face, body and hand landmarks, and gaze and blinking information. Through extensive experiments, we show that our model achieves state-of-the-art results on three key benchmark datasets for depression detection from video by a substantial margin. Our code is publicly available on GitHub.
Code (1)
Tasks
Depression DetectionSimilar Papers 제목 키워드 기반
Mixture of Experts for Recognizing Depression from Interview and Reading Tasks
Depression is a mental disorder and can cause a variety of symptoms, including psychological, physical, and social. Speech has been proved an objective marker for the early recognition of depression. For this reason, man…
Mixture-of-ExpertsCANAMRF: An Attention-Based Model for Multimodal Depression Detection
Multimodal depression detection is an important research topic that aims to predict human mental states using multimodal data. Previous methods treat different modalities equally and fuse each modality by na\"ive mathema…
Depression DetectionIDRL: An Individual-Aware Multimodal Depression-Related Representation Learning Framework for Depression Diagnosis
Depression is a severe mental disorder, and reliable identification plays a critical role in early intervention and treatment. Multimodal depression detection aims to improve diagnostic performance by jointly modeling co…
Representation LearningThe First MPDD Challenge: Multimodal Personality-aware Depression Detection
Depression is a widespread mental health issue affecting diverse age groups, with notable prevalence among college students and the elderly. However, existing datasets and detection methods primarily focus on young adult…
Depression DetectionDepression Diagnosis and Analysis via Multimodal Multi-order Factor Fusion
Depression is a leading cause of death worldwide, and the diagnosis of depression is nontrivial. Multimodal learning is a popular solution for automatic diagnosis of depression, and the existing works suffer two main dra…