Leveraging knowledge distillation for partial multi-task learning from multiple remote sensing datasets
Partial multi-task learning where training examples are annotated for one of the target tasks is a promising idea in remote sensing as it allows combining datasets annotated for different tasks and predicting more tasks with fewer network parameters. The na\"ive approach to partial multi-task learning is sub-optimal due to the lack of all-task annotations for learning joint representations. This paper proposes using knowledge distillation to replace the need of ground truths for the alternate task and enhance the performance of such approach. Experiments conducted on the public ISPRS 2D Semantic Labeling Contest dataset show the effectiveness of the proposed idea on partial multi-task learning for semantic tasks including object detection and semantic segmentation in aerial images.
Code (1)
Tasks
Knowledge DistillationMulti-Task Learningobject-detectionObject DetectionSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generative Diffusion Prior Distillation for Long-Context Knowledge Transfer
While traditional time-series classifiers assume full sequences at inference, practical constraints (latency and cost) often limit inputs to partial prefixes. The absence of class-discriminative patterns in partial data …
Knowledge DistillationQuantifying Knowledge Distillation Using Partial Information Decomposition
Knowledge distillation deploys complex machine learning models in resource-constrained environments by training a smaller student model to emulate internal representations of a complex teacher model. However, the teacher…
Knowledge DistillationTransfer LearningData exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data
Multi-task partially annotated data where each data point is annotated for only a single task are potentially helpful for data scarcity if a network can leverage the inter-task relationship. In this paper, we study the j…
Knowledge DistillationMulti-Task Learningobject-detectionObject Detection+1Life-long Learning for Multilingual Neural Machine Translation with Knowledge Distillation
A common scenario of Multilingual Neural Machine Translation (MNMT) is that each translation task arrives in a sequential manner, and the training data of previous tasks is unavailable. In this scenario, the current meth…
Knowledge DistillationMachine TranslationTranslationToward Fair Graph Neural Networks Via Dual-Teacher Knowledge Distillation
Graph Neural Networks (GNNs) have demonstrated strong performance in graph representation learning across various real-world applications. However, they often produce biased predictions caused by sensitive attributes, su…
FairnessGraph Representation LearningKnowledge DistillationRepresentation Learning