Temporally-Extended Prompts Optimization for SAM in Interactive Medical Image Segmentation
The Segmentation Anything Model (SAM) has recently emerged as a foundation model for addressing image segmentation. Owing to the intrinsic complexity of medical images and the high annotation cost, the medical image segmentation (MIS) community has been encouraged to investigate SAM's zero-shot capabilities to facilitate automatic annotation. Inspired by the extraordinary accomplishments of interactive medical image segmentation (IMIS) paradigm, this paper focuses on assessing the potential of SAM's zero-shot capabilities within the IMIS paradigm to amplify its benefits in the MIS domain. Regrettably, we observe that SAM's vulnerability to prompt forms (e.g., points, bounding boxes) becomes notably pronounced in IMIS. This leads us to develop a framework that adaptively offers suitable prompt forms for human experts. We refer to the framework above as temporally-extended prompts optimization (TEPO) and model it as a Markov decision process, solvable through reinforcement learning. Numerical experiments on the standardized benchmark BraTS2020 demonstrate that the learned TEPO agent can further enhance SAM's zero-shot capability in the MIS context.
Code (0)
등록된 구현이 없습니다.
Tasks
Image SegmentationMedical Image SegmentationSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Assessing Test-time Variability for Interactive 3D Medical Image Segmentation with Diverse Point Prompts
Interactive segmentation model leverages prompts from users to produce robust segmentation. This advancement is facilitated by prompt engineering, where interactive prompts serve as strong priors during test-time. Howeve…
Image SegmentationInteractive SegmentationMedical Image SegmentationPrompt Engineering+3Zero-Shot Gaze-based Volumetric Medical Image Segmentation
Accurate segmentation of anatomical structures in volumetric medical images is crucial for clinical applications, including disease monitoring and cancer treatment planning. Contemporary interactive segmentation models, …
Image SegmentationInteractive SegmentationMedical Image SegmentationSegmentation+2EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria
By simply composing prompts, developers can prototype novel generative applications with Large Language Models (LLMs). To refine prototypes into products, however, developers must iteratively revise prompts by evaluating…
Language ModelingLanguage ModellingLarge Language ModelEfficient Universal Models for Medical Image Segmentation via Weakly Supervised In-Context Learning
Universal models for medical image segmentation, such as interactive and in-context learning (ICL) models, offer strong generalization but require extensive annotations. Interactive models need repeated user prompts for …
Medical Image SegmentationDynamic Prompt Generation for Interactive 3D Medical Image Segmentation Training
Interactive 3D biomedical image segmentation requires efficient models that can iteratively refine predictions based on user prompts. Current foundation models either lack volumetric awareness or suffer from limited inte…
Medical Image Segmentation