Temporal Propagation of Asymmetric Feature Pyramid for Surgical Scene Segmentation
Surgical scene segmentation is crucial for robot-assisted laparoscopic surgery understanding. Current approaches face two challenges: (i) static image limitations including ambiguous local feature similarities and fine-grained structural details, and (ii) dynamic video complexities arising from rapid instrument motion and persistent visual occlusions. While existing methods mainly focus on spatial feature extraction, they fundamentally overlook temporal dependencies in surgical video streams. To address this, we present temporal asymmetric feature propagation network, a bidirectional attention architecture enabling cross-frame feature propagation. The proposed method contains a temporal query propagator that integrates multi-directional consistency constraints to enhance frame-specific feature representation, and an aggregated asymmetric feature pyramid module that preserves discriminative features for anatomical structures and surgical instruments. Our framework uniquely enables both temporal guidance and contextual reasoning for surgical scene understanding. Comprehensive evaluations on two public benchmarks show the proposed method outperforms the current SOTA methods by a large margin, with +16.4\% mIoU on EndoVis2018 and +3.3\% mAP on Endoscapes2023. The code will be publicly available after paper acceptance.
Code (0)
등록된 구현이 없습니다.
Tasks
Scene SegmentationScene UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Surgical Scene Segmentation by Transformer With Asymmetric Feature Enhancement
Surgical scene segmentation is a fundamental task for robotic-assisted laparoscopic surgery understanding. It often contains various anatomical structures and surgical instruments, where similar local textures and fine-g…
AnatomyScene SegmentationScene UnderstandingSegmentationSurgPLAN: Surgical Phase Localization Network for Phase Recognition
Surgical phase recognition is crucial to providing surgery understanding in smart operating rooms. Despite great progress in automatic surgical phase recognition, most existing methods are still restricted by two problem…
Surgical phase recognitionSurgformer: Surgical Transformer with Hierarchical Temporal Attention for Surgical Phase Recognition
Existing state-of-the-art methods for surgical phase recognition either rely on the extraction of spatial-temporal features at a short-range temporal resolution or adopt the sequential extraction of the spatial and tempo…
Surgical phase recognitionDSLO: Deep Sequence LiDAR Odometry Based on Inconsistent Spatio-temporal Propagation
This paper introduces a 3D point cloud sequence learning model based on inconsistent spatio-temporal propagation for LiDAR odometry, termed DSLO. It consists of a pyramid structure with a spatial information reuse strate…
RTETemporally Consistent Label Interpolation for Robust Surgical Multi-Task Learning under Challenging Conditions
Effective multi-task learning for surgical scene understanding is fundamentally hindered by annotation granularity mismatch; temporal workflow tasks such as phase recognition, step recognition and anticipation benefit fr…
Surgical phase recognitionOptical Flow EstimationRepresentation LearningMulti-Task Learning