Modality Fusion Network and Personalized Attention in Momentary Stress Detection in the Wild
Multimodal wearable physiological data in daily life have been used to estimate self-reported stress labels. However, missing data modalities in data collection makes it challenging to leverage all the collected samples. Besides, heterogeneous sensor data and labels among individuals add challenges in building robust stress detection models. In this paper, we proposed a modality fusion network (MFN) to train models and infer self-reported binary stress labels under both complete and incomplete modality conditions. In addition, we applied personalized attention (PA) strategy to leverage personalized representation along with the generalized one-size-fits-all model. We evaluated our methods on a multimodal wearable sensor dataset (N=41) including galvanic skin response (GSR) and electrocardiogram (ECG). Compared to the baseline method using the samples with complete modalities, the performance of the MFN improved by 1.6% in f1-scores. On the other hand, the proposed PA strategy showed a 2.3% higher stress detection f1-score and approximately up to 70% reduction in personalized model parameter size (9.1 MB) compared to the previous state-of-the-art transfer learning strategy (29.3 MB).
Code (1)
Tasks
Transfer LearningSimilar Papers 제목 키워드 기반
Improving Engagement and Efficacy of mHealth Micro-Interventions for Stress Coping: an In-The-Wild Study
Sustaining long-term user engagement with mobile health (mHealth) interventions while preserving their high efficacy remains an ongoing challenge in real-world well-being applications. To address this issue, we introduce…
ChatbotEnhancing Performance and User Engagement in Everyday Stress Monitoring: A Context-Aware Active Reinforcement Learning Approach
In today's fast-paced world, accurately monitoring stress levels is crucial. Sensor-based stress monitoring systems often need large datasets for training effective models. However, individual-specific models are necessa…
Photoplethysmography (PPG)Reinforcement Learning (RL)Learning Longitudinal Stress Dynamics from Irregular Self-Reports via Time Embeddings
The widespread adoption of mobile and wearable sensing technologies has enabled continuous and personalized monitoring of affect, mood disorders, and stress. When combined with ecological self-report questionnaires, thes…
Tracking momentary attention fluctuations with an EEG-based cognitive brain-machine interface
Momentary fluctuations in attention (perceptual accuracy) correlate with neural activity fluctuations in primate visual areas. Yet, the link between such momentary neural fluctuations and attention state remains to be sh…
EEGElectroencephalogram (EEG)SSVEPQuantifying the human visual exposome with vision language models
The visual environment is a fundamental yet unquantified determinant of mental health. While the concept of the environmental exposome is well established, current methods rely on coarse geospatial proxies or biased self…