paper-with-me

홈 › Papers

OCT-SelfNet: A Self-Supervised Framework with Multi-Modal Datasets for Generalized and Robust Retinal Disease Detection

2024-01-22 · Fatema-E Jannat, Sina Gholami, Minhaj Nur Alam, Hamed Tabkhi

Despite the revolutionary impact of AI and the development of locally trained algorithms, achieving widespread generalized learning from multi-modal data in medical AI remains a significant challenge. This gap hinders the practical deployment of scalable medical AI solutions. Addressing this challenge, our research contributes a self-supervised robust machine learning framework, OCT-SelfNet, for detecting eye diseases using optical coherence tomography (OCT) images. In this work, various data sets from various institutions are combined enabling a more comprehensive range of representation. Our method addresses the issue using a two-phase training approach that combines self-supervised pretraining and supervised fine-tuning with a mask autoencoder based on the SwinV2 backbone by providing a solution for real-world clinical deployment. Extensive experiments on three datasets with different encoder backbones, low data settings, unseen data settings, and the effect of augmentation show that our method outperforms the baseline model, Resnet-50 by consistently attaining AUC-ROC performance surpassing 77% across all tests, whereas the baseline model exceeds 54%. Moreover, in terms of the AUC-PR metric, our proposed method exceeded 42%, showcasing a substantial increase of at least 10% in performance compared to the baseline, which exceeded only 33%. This contributes to our understanding of our approach's potential and emphasizes its usefulness in clinical settings.

📄 PDF Abstract BibTeX arXiv:2401.12344

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Multi-OCT-SelfNet: Integrating Self-Supervised Learning with Multi-Source Data Fusion for Enhanced Multi-Class Retinal Disease Classification

2024-09-17 · Fatema-E- Jannat, Sina Gholami, Jennifer I. Lim, Theodore Leng 외

In the medical domain, acquiring large datasets poses significant challenges due to privacy concerns. Nonetheless, the development of a robust deep-learning model for retinal disease diagnosis necessitates a substantial …

Self-Supervised Learning

Contrastive Learning with Cross-Modal Knowledge Mining for Multimodal Human Activity Recognition

2022-05-20 · Razvan Brinzea, Bulat Khaertdinov, Stylianos Asteriadis

Human Activity Recognition is a field of research where input data can take many forms. Each of the possible input modalities describes human behaviour in a different way, and each has its own strengths and weaknesses. W…

Activity RecognitionContrastive LearningHuman Activity RecognitionRetrieval+1

Self-Supervised Multimodal Opinion Summarization

2021-05-27 · ACL 2021 5 · Jinbae Im, Moonki Kim, Hoyeop Lee, Hyunsouk Cho 외

Recently, opinion summarization, which is the generation of a summary from multiple reviews, has been conducted in a self-supervised manner by considering a sampled review as a pseudo summary. However, non-text data such…

DecoderOpinion Summarization

Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes

2022-09-07 · Alex Fedorov, Eloy Geenjaar, Lei Wu, Tristan Sylvain 외

Recent neuroimaging studies that focus on predicting brain disorders via modern machine learning approaches commonly include a single modality and rely on supervised over-parameterized models.However, a single modality p…

DiagnosticSelf-Supervised Learning

Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition

2022-02-24 · ACL 2022 5 · Xichen Pan, Peiyu Chen, Yichen Gong, Helong Zhou 외

Training Transformer-based models demands a large amount of data, while obtaining aligned and labelled data in multimodality is rather cost-demanding, especially for audio-visual speech recognition (AVSR). Thus it makes …

Audio-Visual Speech RecognitionAutomatic Speech Recognition (ASR)Language ModellingLipreading+6