Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images
The generation of three-dimensional (3D) medical images has great application potential since it takes into account the 3D anatomical structure. Two problems prevent effective training of a 3D medical generative model: (1) 3D medical images are expensive to acquire and annotate, resulting in an insufficient number of training images, and (2) a large number of parameters are involved in 3D convolution. Methods: We propose a novel GAN model called 3D Split&Shuffle-GAN. To address the 3D data scarcity issue, we first pre-train a two-dimensional (2D) GAN model using abundant image slices and inflate the 2D convolution weights to improve the initialization of the 3D GAN. Novel 3D network architectures are proposed for both the generator and discriminator of the GAN model to significantly reduce the number of parameters while maintaining the quality of image generation. Several weight inflation strategies and parameter-efficient 3D architectures are investigated. Results: Experiments on both heart (Stanford AIMI Coronary Calcium) and brain (Alzheimer's Disease Neuroimaging Initiative) datasets show that our method leads to improved 3D image generation quality (14.7 improvements on Fr\'echet inception distance) with significantly fewer parameters (only 48.5% of the baseline method). Conclusions: We built a parameter-efficient 3D medical image generation model. Due to the efficiency and effectiveness, it has the potential to generate high-quality 3D brain and heart images for real use cases.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationMedical Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing
Background and objective: Sharing of medical data is required to enable the cross-agency flow of healthcare information and construct high-accuracy computer-aided diagnosis systems. However, the large sizes of medical da…
Dataset DistillationImage GenerationvalidShaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging
The generalization of the Transformer architecture via MetaFormer has reshaped our understanding of its success in computer vision. By replacing self-attention with simpler token mixers, MetaFormer provides strong baseli…
Semantic SegmentationImage ClassificationContinuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks
Facial expressions are one of the most powerful ways for depicting specific patterns in human behavior and describing human emotional state. Despite the impressive advances of affective computing over the last decade, au…
Emotion RecognitionFacial Expression RecognitionFacial Expression Recognition (FER)Transfer LearningCNNs and GANs in MRI-based cross-modality medical image estimation
Cross-modality image estimation involves the generation of images of one medical imaging modality from that of another modality. Convolutional neural networks (CNNs) have been shown to be useful in identifying, character…
Reinventing 2D Convolutions for 3D Images
There have been considerable debates over 2D and 3D representation learning on 3D medical images. 2D approaches could benefit from large-scale 2D pretraining, whereas they are generally weak in capturing large 3D context…
Representation Learning