Semi-supervised Multi-task Learning for Semantics and Depth
Multi-Task Learning (MTL) aims to enhance the model generalization by sharing representations between related tasks for better performance. Typical MTL methods are jointly trained with the complete multitude of ground-truths for all tasks simultaneously. However, one single dataset may not contain the annotations for each task of interest. To address this issue, we propose the Semi-supervised Multi-Task Learning (SemiMTL) method to leverage the available supervisory signals from different datasets, particularly for semantic segmentation and depth estimation tasks. To this end, we design an adversarial learning scheme in our semi-supervised training by leveraging unlabeled data to optimize all the task branches simultaneously and accomplish all tasks across datasets with partial annotations. We further present a domain-aware discriminator structure with various alignment formulations to mitigate the domain discrepancy issue among datasets. Finally, we demonstrate the effectiveness of the proposed method to learn across different datasets on challenging street view and remote sensing benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationMulti-Task LearningSemantic SegmentationSimilar Papers 제목 키워드 기반
Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth
Multi-task learning (MTL) paradigm focuses on jointly learning two or more tasks, aiming for significant improvement w.r.t model's generalizability, performance, and training/inference memory footprint. The aforementione…
Data AugmentationDepth EstimationMulti-Task LearningSegmentation+1Semi-Supervised Semantic Depth Estimation using Symbiotic Transformer and NearFarMix Augmentation
In computer vision, depth estimation is crucial for domains like robotics, autonomous vehicles, augmented reality, and virtual reality. Integrating semantics with depth enhances scene understanding through reciprocal inf…
Autonomous VehiclesDepth EstimationScene UnderstandingSemantic SegmentationGeometry meets semantics for semi-supervised monocular depth estimation
Depth estimation from a single image represents a very exciting challenge in computer vision. While other image-based depth sensing techniques leverage on the geometry between different viewpoints (e.g., stereo or struct…
DecoderDepth EstimationDepth PredictionMonocular Depth Estimation+1DepthMatch: Semi-Supervised RGB-D Scene Parsing through Depth-Guided Regularization
RGB-D scene parsing methods effectively capture both semantic and geometric features of the environment, demonstrating great potential under challenging conditions such as extreme weather and low lighting. However, exist…
Scene ParsingSemantic SegmentationXVO: Generalized Visual Odometry via Cross-Modal Self-Training
We propose XVO, a semi-supervised learning method for training generalized monocular Visual Odometry (VO) models with robust off-the-self operation across diverse datasets and settings. In contrast to standard monocular …
Monocular Visual OdometryMotion EstimationTransfer LearningVisual Odometry