Papers Multimodal Deep Learning
“Multimodal Deep Learning” 태그가 달린 논문 297편 · 필터 해제
A Multimodal Deep Learning Framework for Edema Classification Using HCT and Clinical Data
We propose AttentionMixer, a unified deep learning framework for multimodal detection of brain edema that combines structural head CT (HCT) with routine clinical metadata. While HCT provides rich spatial information, cli…
Multimodal Deep LearningMultimodal Deep Learning for Early Prediction of Patient Deterioration in the ICU: Integrating Time-Series EHR Data with Clinical Notes
Early identification of patients at risk for clinical deterioration in the intensive care unit (ICU) remains a critical challenge. Delayed recognition of impending adverse events, including mortality, vasopressor initiat…
Multimodal Deep LearningLearning from Limited and Incomplete Data: A Multimodal Framework for Predicting Pathological Response in NSCLC
Major pathological response (pR) following neoadjuvant therapy is a clinically meaningful endpoint in non-small cell lung cancer, strongly associated with improved survival. However, accurate preoperative prediction of p…
Multimodal Deep LearningComplementarity-Preserving Generative Theory for Multimodal ECG Synthesis: A Quantum-Inspired Approach
Multimodal deep learning has substantially improved electrocardiogram (ECG) classification by jointly leveraging time, frequency, and time-frequency representations. However, existing generative models typically synthesi…
Multimodal Deep LearningMultimodal classification of Radiation-Induced Contrast Enhancements and tumor recurrence using deep learning
The differentiation between tumor recurrence and radiation-induced contrast enhancements in post-treatment glioblastoma patients remains a major clinical challenge. Existing approaches rely on clinically sparsely availab…
Multimodal Deep LearningMultimodal Deep Learning for Dynamic and Static Neuroimaging: Integrating MRI and fMRI for Alzheimer Disease Analysis
Magnetic Resonance Imaging (MRI) provides detailed structural information, while functional MRI (fMRI) captures temporal brain activity. In this work, we present a multimodal deep learning framework that integrates MRI a…
Multi-class ClassificationMultimodal Deep LearningData AugmentationArt2Mus: Artwork-to-Music Generation via Visual Conditioning and Large-Scale Cross-Modal Alignment
Music generation has advanced markedly through multimodal deep learning, enabling models to synthesize audio from text and, more recently, from images. However, existing image-conditioned systems suffer from two fundamen…
Multimodal Deep LearningMusic GenerationTowards Affordable, Non-Invasive Real-Time Hypoglycemia Detection Using Wearable Sensor Signals
Accurately detecting hypoglycemia without invasive glucose sensors remains a critical challenge in diabetes management, particularly in regions where continuous glucose monitoring (CGM) is prohibitively expensive or clin…
Multimodal Deep LearningA Unified Multimodal Framework for Dataset Construction and Model-Based Diagnosis of Ameloblastoma
Artificial intelligence (AI)-enabled diagnostics in maxillofacial pathology require structured, high-quality multimodal datasets. However, existing resources provide limited ameloblastoma coverage and lack the format con…
Multimodal Deep LearningInference-Time Dynamic Modality Selection for Incomplete Multimodal Classification
Multimodal deep learning (MDL) has achieved remarkable success across various domains, yet its practical deployment is often hindered by incomplete multimodal data. Existing incomplete MDL methods either discard missing …
Multimodal Deep LearningRepresentation LearningPovNet+: A Deep Learning Architecture for Socially Assistive Robots to Learn and Assist with Multiple Activities of Daily Living
A significant barrier to the long-term deployment of autonomous socially assistive robots is their inability to both perceive and assist with multiple activities of daily living (ADLs). In this paper, we present the firs…
Human Activity RecognitionMultimodal Deep LearningIceWatch: Forecasting Glacial Lake Outburst Floods (GLOFs) using Multimodal Deep Learning
Glacial Lake Outburst Floods (GLOFs) pose a serious threat in high mountain regions. They are hazardous to communities, infrastructure, and ecosystems further downstream. The classical methods of GLOF detection and predi…
Multimodal Deep LearningDigital FAST: An AI-Driven Multimodal Framework for Rapid and Early Stroke Screening
Early identification of stroke symptoms is essential for enabling timely intervention and improving patient outcomes, particularly in prehospital settings. This study presents a fast, non-invasive multimodal deep learnin…
Multimodal Deep LearningTransfer LearningModel selection and real-time skill assessment for suturing in robotic surgery
Automated feedback systems have the potential to provide objective skill assessment for training and evaluation in robot-assisted surgery. In this study, we examine methods to achieve real-time prediction of surgical ski…
Multimodal Deep LearningHandling Missing Modalities in Multimodal Survival Prediction for Non-Small Cell Lung Cancer
Accurate survival prediction in Non-Small Cell Lung Cancer (NSCLC) requires integrating clinical, radiological, and histopathological data. Multimodal Deep Learning (MDL) can improve precision prognosis, but small cohort…
Multimodal Deep LearningA Multimodal Deep Learning Framework for Predicting ICU Deterioration: Integrating ECG Waveforms with Clinical Data and Clinician Benchmarking
Artificial intelligence holds strong potential to support clinical decision making in intensive care units where timely and accurate risk assessment is critical. However, many existing models focus on isolated outcomes o…
Multimodal Deep LearningDecision MakingA Spatially Masked Adaptive Gated Network for multimodal post-flood water extent mapping using SAR and incomplete multispectral data
Mapping water extent during a flood event is essential for effective disaster management throughout all phases: mitigation, preparedness, response, and recovery. In particular, during the response stage, when timely and …
Multimodal Deep LearningPlacenta Accreta Spectrum Detection using Multimodal Deep Learning
Placenta Accreta Spectrum (PAS) is a life-threatening obstetric complication involving abnormal placental invasion into the uterine wall. Early and accurate prenatal diagnosis is essential to reduce maternal and neonatal…
Multimodal Deep LearningFETAL-GAUGE: A Benchmark for Assessing Vision-Language Models in Fetal Ultrasound
The growing demand for prenatal ultrasound imaging has intensified a global shortage of trained sonographers, creating barriers to essential fetal health monitoring. Deep learning has the potential to enhance sonographer…
Visual Question AnsweringMultimodal Deep LearningVisual GroundingRadiology Report Generation with Layer-Wise Anatomical Attention
Automatic radiology report generation is a promising application of multimodal deep learning, aiming to reduce reporting workload and improve consistency. However, current state-of-the-art (SOTA) systems - such as Multim…
Multimodal Deep Learning