NAPS: Attention-Based Fusion of Heterogeneous Physiological Signals
Physiological signals are inherently heterogeneous: they are collected under diverse acquisition setups, differ in the number and type of modalities and channels, varying in quality, reliability, and relevance across tasks. This variability poses a major challenge for machine learning models required to generalize across subjects, sensors, and clinical environments. Existing approaches typically train on limited modalities or single channels, leading to marginal representations that, on their own, fail to capture the systemic complexity of the physiological state; naive fusion of such representations, such as via pooling or voting schemes, is typically suboptimal, as it cannot adaptively weight different sources or capture temporal, spatial, and cross-modality dependencies. We introduce NAPS (Neural Aggregator of Physiological Signals), a neural module that performs principled data fusion to derive unified physiological representations, employing an ad hoc tri-axial attention mechanism and dimension-adaptive training to robustly manage varying high-dimensional sensor configurations. We test NAPS on automatic sleep staging from polysomnography (PSG), an ideal real-world application, where recordings consist of multiple physiological signals (EEG, EOG, EMG, ...), considerably varying in configuration across datasets and institutions. Leveraging frozen pretrained unimodal encoders, NAPS dynamically integrates representations or predictions, achieving state-of-the-art generalization across multiple datasets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
H2G2-Net: A Hierarchical Heterogeneous Graph Generative Network Framework for Discovery of Multi-Modal Physiological Responses
Discovering human cognitive and emotional states using multi-modal physiological signals draws attention across various research applications. Physiological responses of the human body are influenced by human cognition a…
Emotion recognition based on multi-modal electrophysiology multi-head attention Contrastive Learning
Emotion recognition is an important research direction in artificial intelligence, helping machines understand and adapt to human emotional states. Multimodal electrophysiological(ME) signals, such as EEG, GSR, respirati…
Contrastive LearningEEGEmotion RecognitionMultimodal Emotion RecognitionCombining Facial Videos and Biosignals for Stress Estimation During Driving
Reliable stress recognition is critical in applications such as medical monitoring and safety-critical systems, including real-world driving. While stress is commonly detected using physiological signals such as perinasa…
Multimodal Latent Emotion Recognition from Micro-expression and Physiological Signals
This paper discusses the benefits of incorporating multimodal data for improving latent emotion recognition accuracy, focusing on micro-expression (ME) and physiological signals (PS). The proposed approach presents a nov…
Emotion RecognitionDeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks
Non-contact video-based physiological measurement has many applications in health care and human-computer interaction. Practical applications require measurements to be accurate even in the presence of large head rotatio…
Motion Estimation