MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation
We propose Masked-Attention Transformers for Surgical Instrument Segmentation (MATIS), a two-stage, fully transformer-based method that leverages modern pixel-wise attention mechanisms for instrument segmentation. MATIS exploits the instance-level nature of the task by employing a masked attention module that generates and classifies a set of fine instrument region proposals. Our method incorporates long-term video-level information through video transformers to improve temporal consistency and enhance mask classification. We validate our approach in the two standard public benchmarks, Endovis 2017 and Endovis 2018. Our experiments demonstrate that MATIS' per-frame baseline outperforms previous state-of-the-art methods and that including our temporal consistency module boosts our model's performance further.
Code (1)
Tasks
SegmentationSimilar Papers 제목 키워드 기반
SurgMAE: Masked Autoencoders for Long Surgical Video Analysis
There has been a growing interest in using deep learning models for processing long surgical videos, in order to automatically detect clinical/operational activities and extract metrics that can enable workflow efficienc…
Self-Supervised LearningTowards Holistic Surgical Scene Understanding
Most benchmarks for studying surgical interventions focus on a specific challenge instead of leveraging the intrinsic complementarity among different tasks. In this work, we present a new experimental framework towards h…
Action RecognitionAtomic action recognitionScene UnderstandingSurgical phase recognitionRAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments
Semantic segmentation of surgical instruments plays a crucial role in robot-assisted surgery. However, accurate segmentation of cataract surgical instruments is still a challenge due to specular reflection and class imba…
SegmentationSemantic SegmentationAttention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments
The real-time segmentation of surgical instruments plays a crucial role in robot-assisted surgery. However, it is still a challenging task to implement deep learning models to do real-time segmentation for surgical instr…
DecoderSurgeoNet: Realtime 3D Pose Estimation of Articulated Surgical Instruments from Stereo Images using a Synthetically-trained Network
Surgery monitoring in Mixed Reality (MR) environments has recently received substantial focus due to its importance in image-based decisions, skill assessment, and robot-assisted surgery. Tracking hands and articulated s…
3D Pose EstimationMixed RealityPose Estimation