Generative Diffusion Model Bootstraps Zero-shot Classification of Fetal Ultrasound Images In Underrepresented African Populations

2024-07-29 · Fangyijie Wang, Kevin Whelan, Guénolé Silvestre, Kathleen M. Curran

Developing robust deep learning models for fetal ultrasound image analysis requires comprehensive, high-quality datasets to effectively learn informative data representations within the domain. However, the scarcity of labelled ultrasound images poses substantial challenges, especially in low-resource settings. To tackle this challenge, we leverage synthetic data to enhance the generalizability of deep learning models. This study proposes a diffusion-based method, Fetal Ultrasound LoRA (FU-LoRA), which involves fine-tuning latent diffusion models using the LoRA technique to generate synthetic fetal ultrasound images. These synthetic images are integrated into a hybrid dataset that combines real-world and synthetic images to improve the performance of zero-shot classifiers in low-resource settings. Our experimental results on fetal ultrasound images from African cohorts demonstrate that FU-LoRA outperforms the baseline method by a 13.73% increase in zero-shot classification accuracy. Furthermore, FU-LoRA achieves the highest accuracy of 82.40%, the highest F-score of 86.54%, and the highest AUC of 89.78%. It demonstrates that the FU-LoRA method is effective in the zero-shot classification of fetal ultrasound images in low-resource settings. Our code and data are publicly accessible on https://github.com/13204942/FU-LoRA.

📄 PDF Abstract BibTeX arXiv:2407.20072

Code (1)

13204942/fu-lora 공식 구현 pytorch

Tasks

zero-shot-classificationZero-Shot Learning

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

2023-01-30 · Conference 2023 2 · Junnan Li, Dongxu Li, Silvio Savarese, Steven Hoi

The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models. This paper proposes BLIP-2, a generic and efficient pre-training strategy that bootstraps…

Generative Visual Question Answering Image Captioning Image Retrieval Image to text+13

A Simple and Efficient Baseline for Zero-Shot Generative Classification

2024-12-17 · Zipeng Qi, Buhua Liu, Shiyan Zhang, Bao Li 외

Large diffusion models have become mainstream generative models in both academic studies and industrial AIGC applications. Recently, a number of works further explored how to employ the power of large diffusion models as…

zero-shot-classification Zero-Shot Learning

Diffusion in Zero-Shot Learning for Environmental Audio

2024-12-04 · Ysobel Sims, Stephan Chalup, Alexandre Mendes

Zero-shot learning enables models to generalize to unseen classes by leveraging semantic information, bridging the gap between training and testing sets with non-overlapping classes. While much research has focused on ze…

Generative Adversarial Network Zero-Shot Learning

Text-to-Image Diffusion Models are Zero-Shot Classifiers

2023-03-27 · Kevin Clark, Priyank Jaini

The excellent generative capabilities of text-to-image diffusion models suggest they learn informative representations of image-text data. However, what knowledge their representations capture is not fully understood, an…

Attribute Contrastive Learning image-classification Image Classification+1

Text-to-Image Diffusion Models are Zero Shot Classifiers

2023-09-21 · NeurIPS 2023 11