Noisy-LSTM: Improving Temporal Awareness for Video Semantic Segmentation
Semantic video segmentation is a key challenge for various applications. This paper presents a new model named Noisy-LSTM, which is trainable in an end-to-end manner, with convolutional LSTMs (ConvLSTMs) to leverage the temporal coherency in video frames. We also present a simple yet effective training strategy, which replaces a frame in video sequence with noises. This strategy spoils the temporal coherency in video frames during training and thus makes the temporal links in ConvLSTMs unreliable, which may consequently improve feature extraction from video frames, as well as serve as a regularizer to avoid overfitting, without requiring extra data annotation or computational costs. Experimental results demonstrate that the proposed model can achieve state-of-the-art performances in both the CityScapes and EndoVis2018 datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic SegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Mobile Video Object Detection with Temporally-Aware Feature Maps
This paper introduces an online model for object detection in videos designed to run in real-time on low-powered mobile and embedded devices. Our approach combines fast single-image object detection with convolutional lo…
CPUObjectobject-detectionObject Detection+1Leveraging Weak Semantic Relevance for Complex Video Event Classification
Existing video event classification approaches suffer from limited human-labeled semantic annotations. Weak semantic annotations can be harvested from Web-knowledge without involving any human interaction. However such w…
ClassificationGeneral ClassificationFuture Semantic Segmentation with Convolutional LSTM
We consider the problem of predicting semantic segmentation of future frames in a video. Given several observed frames in a video, our goal is to predict the semantic segmentation map of future frames that are not yet ob…
Autonomous DrivingDecision MakingFuture predictionSegmentation+1STFCN: Spatio-Temporal FCN for Semantic Video Segmentation
This paper presents a novel method to involve both spatial and temporal features for semantic video segmentation. Current work on convolutional neural networks(CNNs) has shown that CNNs provide advanced spatial features …
SegmentationSemantic SegmentationVideo SegmentationVideo Semantic SegmentationStimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models
Predicting and reasoning how a video would make a human feel is crucial for developing socially intelligent systems. Although Multimodal Large Language Models (MLLMs) have shown impressive video understanding capabilitie…
Video Understanding