Metrics Revolutions: Groundbreaking Insights into the Implementation of Metrics for Biomedical Image Segmentation
The evaluation of segmentation performance is a common task in biomedical image analysis, with its importance emphasized in the recently released metrics selection guidelines and computing frameworks. To quantitatively evaluate the alignment of two segmentations, researchers commonly resort to counting metrics, such as the Dice similarity coefficient, or distance-based metrics, such as the Hausdorff distance, which are usually computed by publicly available open-source tools with an inherent assumption that these tools provide consistent results. In this study we questioned this assumption, and performed a systematic implementation analysis along with quantitative experiments on real-world clinical data to compare 11 open-source tools for distance-based metrics computation against our highly accurate mesh-based reference implementation. The results revealed that statistically significant differences among all open-source tools are both surprising and concerning, since they question the validity of existing studies. Besides identifying the main sources of variation, we also provide recommendations for distance-based metrics computation.
Code (1)
Tasks
Image SegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
RL-Scope: Cross-Stack Profiling for Deep Reinforcement Learning Workloads
Deep reinforcement learning (RL) has made groundbreaking advancements in robotics, data center management and other applications. Unfortunately, system-level bottlenecks in RL workloads are poorly understood; we observe …
CPUDeep Reinforcement LearningGPUManagement+3Exponential scaling of neural algorithms - a future beyond Moore's Law?
Although the brain has long been considered a potential inspiration for future computing, Moore's Law - the scaling property that has seen revolutions in technologies ranging from supercomputers to smart phones - has lar…
DNA Fragments in Crude Oil Reveals Earth's Hidden History
This groundbreaking research extracted DNA from petroleum using nanoparticle affinity bead technology, yielding 3,159,020 petroleum DNA (pDNA) sequences, primarily environmental DNA. While most original in situ DNA (oriD…
Privacy-Preserving Generative Models: A Comprehensive Survey
Despite the generative model's groundbreaking success, the need to study its implications for privacy and utility becomes more urgent. Although many studies have demonstrated the privacy threats brought by GANs, no exist…
Privacy PreservingSurveyBeyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures
The enduring legacy of Euclidean geometry underpins classical machine learning, which, for decades, has been primarily developed for data lying in Euclidean space. Yet, modern machine learning increasingly encounters ric…