Learning to ground medical text in a 3D human atlas
In this paper, we develop a method for grounding medical text into a physically meaningful and interpretable space corresponding to a human atlas. We build on text embedding architectures such as Bert and introduce a loss function that allows us to reason about the semantic and spatial relatedness of medical texts by learning a projection of the embedding into a 3D space representing the human body. We quantitatively and qualitatively demonstrate that our proposed method learns a context sensitive and spatially aware mapping, in both the inter-organ and intra-organ sense, using a large scale medical text dataset from the {``}Large-scale online biomedical semantic indexing{''} track of the 2020 BioASQ challenge. We extend our approach to a self-supervised setting, and find it to be competitive with a classification based method, and a fully supervised variant of approach.
Code (1)
Tasks
Phrase GroundingVisual GroundingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
LT-Net: Label Transfer by Learning Reversible Voxel-wise Correspondence for One-shot Medical Image Segmentation
We introduce a one-shot segmentation method to alleviate the burden of manual annotation for medical images. The main idea is to treat one-shot segmentation as a classical atlas-based segmentation problem, where voxel-wi…
Image SegmentationMedical Image SegmentationOne-Shot SegmentationSegmentation+1Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation
Accurate segmentation of anatomical structures in medical images is essential for diagnosis and treatment planning. While recent interactive segmentation foundation models enhance generalization through large-scale multi…
Medical Image SegmentationInteractive SegmentationSelf-supervised context-aware COVID-19 document exploration through atlas grounding
In this paper, we aim to develop a self-supervised grounding of Covid-related medical text based on the actual spatial relationships between the referred anatomical concepts. More specifically, we learn to project senten…
RetrievalMedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text
Artificial intelligence has demonstrated significant potential in clinical decision-making; however, developing models capable of adapting to diverse real-world scenarios and performing complex diagnostic reasoning remai…
Question AnsweringMapping the Human Body at Cellular Resolution -- The NIH Common Fund Human BioMolecular Atlas Program
Transformative technologies are enabling the construction of three dimensional (3D) maps of tissues with unprecedented spatial and molecular resolution. Over the next seven years, the NIH Common Fund Human Biomolecular A…