Shift-Memory Network for Temporal Scene Segmentation
Semantic segmentation has achieved great accuracy in understanding spatial layout. For real-time tasks based on dynamic scenes, we extend semantic segmentation in temporal domain to enhance the spatial accuracy with motion. We utilize a shift-mode network over streaming input to ensure zero-latency output. For the data overlap under shifting network, this paper identifies repeated computation in fixed periods across network layers. To avoid this redundancy, we derive a Shift-Memory Network (SMN) from encoding-decoding baseline to reuse the network values without accuracy loss. Trained in patch-mode, the SMN extracts the network parameters for SMN to perform inference promptly in compact memory. We segment dynamic scenes from 1D scanning input and 2D video. The experiments of SMN achieve equivalent accuracy as shift-mode but in faster inference speeds and much smaller memory. This will facilitate semantic segmentation in real-time application on edge devices.
Code (0)
등록된 구현이 없습니다.
Tasks
Scene SegmentationSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Class-Agnostic Visio-Temporal Scene Sketch Semantic Segmentation
Scene sketch semantic segmentation is a crucial task for various applications including sketch-to-image retrieval and scene understanding. Existing sketch segmentation methods treat sketches as bitmap images, leading to …
Image RetrievalScene UnderstandingSegmentationSemantic SegmentationExploiting Temporal State Space Sharing for Video Semantic Segmentation
Video semantic segmentation (VSS) plays a vital role in understanding the temporal evolution of scenes. Traditional methods often segment videos frame-by-frame or in a short temporal window, leading to limited temporal c…
MambaSemantic SegmentationState Space ModelsVideo Semantic SegmentationMemory Matters: Convolutional Recurrent Neural Network for Scene Text Recognition
Text recognition in natural scene is a challenging problem due to the many factors affecting text appearance. In this paper, we presents a method that directly transcribes scene text images to text without needing of sop…
Scene Text RecognitionSegmentationTEDi: Temporal Memory-Enhanced and Denoising Transformer for Surgical Instrument Segmentation
Query-based segmentation methods have shown promising potential for surgical instrument segmentation and recognition, which is essential for scene understanding and downstream tasks in computer assisted surgery. However,…
Scene UnderstandingMemorySeg: Online LiDAR Semantic Segmentation with a Latent Memory
Semantic segmentation of LiDAR point clouds has been widely studied in recent years, with most existing methods focusing on tackling this task using a single scan of the environment. However, leveraging the temporal stre…
LIDAR Semantic SegmentationSegmentationSemantic Segmentation