Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation
Most research on novel techniques for 3D Medical Image Segmentation (MIS) is currently done using Deep Learning with GPU accelerators. The principal challenge of such technique is that a single input can easily cope computing resources, and require prohibitive amounts of time to be processed. Distribution of deep learning and scalability over computing devices is an actual need for progressing on such research field. Conventional distribution of neural networks consist in data parallelism, where data is scattered over resources (e.g., GPUs) to parallelize the training of the model. However, experiment parallelism is also an option, where different training processes are parallelized across resources. While the first option is much more common on 3D image segmentation, the second provides a pipeline design with less dependence among parallelized processes, allowing overhead reduction and more potential scalability. In this work we present a design for distributed deep learning training pipelines, focusing on multi-node and multi-GPU environments, where the two different distribution approaches are deployed and benchmarked. We take as proof of concept the 3D U-Net architecture, using the MSD Brain Tumor Segmentation dataset, a state-of-art problem in medical image segmentation with high computing and space requirements. Using the BSC MareNostrum supercomputer as benchmarking environment, we use TensorFlow and Ray as neural network training and experiment distribution platforms. We evaluate the experiment speed-up, showing the potential for scaling out on GPUs and nodes. Also comparing the different parallelism techniques, showing how experiment distribution leverages better such resources through scaling. Finally, we provide the implementation of the design open to the community, and the non-trivial steps and methodology for adapting and deploying a MIS case as the here presented.
Code (1)
Tasks
BenchmarkingBrain Tumor SegmentationDeep LearningGPUImage SegmentationMedical Image SegmentationSegmentationSemantic SegmentationTumor SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation
Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. The inherent privacy-preserving property of FL algorithms makes them esp…
Federated LearningHyperparameter OptimizationImage SegmentationLesion Segmentation+6Performance Analysis of UNet and Variants for Medical Image Segmentation
Medical imaging plays a crucial role in modern healthcare by providing non-invasive visualisation of internal structures and abnormalities, enabling early disease detection, accurate diagnosis, and treatment planning. Th…
Image SegmentationMedical Image SegmentationSegmentationSemantic SegmentationHyperparameter-Free Medical Image Synthesis for Sharing Data and Improving Site-Specific Segmentation
Sharing synthetic medical images is a promising alternative to sharing real images that can improve patient privacy and data security. To get good results, existing methods for medical image synthesis must be manually ad…
Image GenerationTuneVLSeg: Prompt Tuning Benchmark for Vision-Language Segmentation Models
Vision-Language Models (VLMs) have shown impressive performance in vision tasks, but adapting them to new domains often requires expensive fine-tuning. Prompt tuning techniques, including textual, visual, and multimodal …
BenchmarkingSegmentationVision-Language SegmentationVisual Prompt TuningSegNAS3D: Network Architecture Search with Derivative-Free Global Optimization for 3D Image Segmentation
Deep learning has largely reduced the need for manual feature selection in image segmentation. Nevertheless, network architecture optimization and hyperparameter tuning are mostly manual and time consuming. Although ther…
feature selectionglobal-optimizationimage-classificationImage Classification+4