Multi-Scale and Multi-Modal Contrastive Learning Network for Biomedical Time Series
Multi-modal biomedical time series (MBTS) data offers a holistic view of the physiological state, holding significant importance in various bio-medical applications. Owing to inherent noise and distribution gaps across different modalities, MBTS can be complex to model. Various deep learning models have been developed to learn representations of MBTS but still fall short in robustness due to the ignorance of modal-to-modal variations. This paper presents a multi-scale and multi-modal biomedical time series representation learning (MBSL) network with contrastive learning to migrate these variations. Firstly, MBTS is grouped based on inter-modal distances, then each group with minimum intra-modal variations can be effectively modeled by individual encoders. Besides, to enhance the multi-scale feature extraction (encoder), various patch lengths and mask ratios are designed to generate tokens with semantic information at different scales and diverse contextual perspectives respectively. Finally, cross-modal contrastive learning is proposed to maximize consistency among inter-modal groups, maintaining useful information and eliminating noises. Experiments against four bio-medical applications show that MBSL outperforms state-of-the-art models by 33.9% mean average errors (MAE) in respiration rate, by 13.8% MAE in exercise heart rate, by 1.41% accuracy in human activity recognition, and by 1.14% F1-score in obstructive sleep apnea-hypopnea syndrome.
Code (0)
등록된 구현이 없습니다.
Tasks
Activity RecognitionContrastive LearningHuman Activity RecognitionRepresentation LearningTime SeriesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Thin Bridges for Drug Text Alignment: Lightweight Contrastive Learning for Target Specific Drug Retrieval
Multimodal foundation models hold promise for drug discovery and biomedical applications, but most existing approaches rely on heavy pretraining or large scale multimodal corpora. We investigate whether thin contrastive …
Contrastive LearningDrug DiscoveryCan representation learning for multimodal image registration be improved by supervision of intermediate layers?
Multimodal imaging and correlative analysis typically require image alignment. Contrastive learning can generate representations of multimodal images, reducing the challenging task of multimodal image registration to a m…
Contrastive Learningimage-classificationImage ClassificationImage Registration+2Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs
Biomedical Knowledge Graphs (BKGs) integrate diverse datasets to elucidate complex relationships within the biomedical field. Effective link prediction on these graphs can uncover valuable connections, such as potential …
Contrastive LearningGraph EmbeddingKnowledge Graph EmbeddingKnowledge Graphs+2Contrastive Regularization over LoRA for Multimodal Biomedical Image Incremental Learning
Multimodal Biomedical Image Incremental Learning (MBIIL) is essential for handling diverse tasks and modalities in the biomedical domain, as training separate models for each modality or task significantly increases infe…
Computational EfficiencyIncremental LearningBiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs
Biomedical data is inherently multimodal, comprising physical measurements and natural language narratives. A generalist biomedical AI model needs to simultaneously process different modalities of data, including text an…
ArticlesMedical Visual Question AnsweringPneumonia DetectionQuestion Answering+3