Optical Flow augmented Semantic Segmentation networks for Automated Driving
Motion is a dominant cue in automated driving systems. Optical flow is typically computed to detect moving objects and to estimate depth using triangulation. In this paper, our motivation is to leverage the existing dense optical flow to improve the performance of semantic segmentation. To provide a systematic study, we construct four different architectures which use RGB only, flow only, RGBF concatenated and two-stream RGB + flow. We evaluate these networks on two automotive datasets namely Virtual KITTI and Cityscapes using the state-of-the-art flow estimator FlowNet v2. We also make use of the ground truth optical flow in Virtual KITTI to serve as an ideal estimator and a standard Farneback optical flow algorithm to study the effect of noise. Using the flow ground truth in Virtual KITTI, two-stream architecture achieves the best results with an improvement of 4% IoU. As expected, there is a large improvement for moving objects like trucks, vans and cars with 38%, 28% and 6% increase in IoU. FlowNet produces an improvement of 2.4% in average IoU with larger improvement in the moving objects corresponding to 26%, 11% and 5% in trucks, vans and cars. In Cityscapes, flow augmentation provided an improvement for moving objects like motorcycle and train with an increase of 17% and 7% in IoU.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationSemantic SegmentationSimilar Papers 제목 키워드 기반
Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow
A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from…
Optical Flow EstimationSegmentationSemantic SegmentationVideo Segmentation+1Joint Optical Flow and Temporally Consistent Semantic Segmentation
The importance and demands of visual scene understanding have been steadily increasing along with the active development of autonomous systems. Consequently, there has been a large amount of research dedicated to semanti…
Motion EstimationOptical Flow EstimationScene UnderstandingSegmentation+1Video Semantic Segmentation with Inter-Frame Feature Fusion and Inner-Frame Feature Refinement
Video semantic segmentation aims to generate accurate semantic maps for each video frame. To this end, many works dedicate to integrate diverse information from consecutive frames to enhance the features for prediction, …
Optical Flow EstimationSemantic SegmentationVideo Semantic SegmentationOptical Flow with Semantic Segmentation and Localized Layers
Existing optical flow methods make generic, spatially homogeneous, assumptions about the spatial structure of the flow. In reality, optical flow varies across an image depending on object class. Simply put, different obj…
Optical Flow EstimationScene SegmentationSegmentationSemantic SegmentationUnsupervised Domain Adaptation by Optical Flow Augmentation in Semantic Segmentation
It is expensive to generate real-life image labels and there is a domain gap between real-life and simulated images, hence a model trained on the latter cannot adapt to the former. Solving this can totally eliminate the …
Domain AdaptationOptical Flow EstimationSegmentationSemantic Segmentation+1