GazeSAM: What You See is What You Segment
This study investigates the potential of eye-tracking technology and the Segment Anything Model (SAM) to design a collaborative human-computer interaction system that automates medical image segmentation. We present the \textbf{GazeSAM} system to enable radiologists to collect segmentation masks by simply looking at the region of interest during image diagnosis. The proposed system tracks radiologists' eye movement and utilizes the eye-gaze data as the input prompt for SAM, which automatically generates the segmentation mask in real time. This study is the first work to leverage the power of eye-tracking technology and SAM to enhance the efficiency of daily clinical practice. Moreover, eye-gaze data coupled with image and corresponding segmentation labels can be easily recorded for further advanced eye-tracking research. The code is available in \url{https://github.com/ukaukaaaa/GazeSAM}.
Code (1)
Tasks
Image SegmentationMedical Image SegmentationSegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
What Makes for Automatic Reconstruction of Pulmonary Segments
3D reconstruction of pulmonary segments plays an important role in surgical treatment planning of lung cancer, which facilitates preservation of pulmonary function and helps ensure low recurrence rates. However, automati…
3D ReconstructionTraffNet: Learning Causality of Traffic Generation for What-if Prediction
Real-time what-if traffic prediction is crucial for decision making in intelligent traffic management and control. Although current deep learning methods demonstrate significant advantages in traffic prediction, they are…
Decision MakingManagementPredictionTraffic PredictionWhat Matters Most in Morphologically Segmented SMT Models?
What-Where Transformer: A Slot-Centric Visual Backbone for Concurrent Representation and Localization
Many image understanding tasks involve identifying what is present and where it appears. However, tasks that address where, such as object discovery, detection, and segmentation, are often considerably more complex than …
Semantic SegmentationImage ClassificationComposing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Models
There have recently been large advances both in pre-training visual representations for robotic control and segmenting unknown category objects in general images. To leverage these for improved robot learning, we propose…
ObjectSystematic Generalization