paper-with-me

Papers

Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation

2023-09-22 · Ping Li, Junjie Chen, Li Yuan, Xianghua Xu, Mingli Song

To alleviate the expensive human labeling, semi-supervised semantic segmentation employs a few labeled images and an abundant of unlabeled images to predict the pixel-level label map with the same size. Previous methods often adopt co-training using two convolutional networks with the same architecture but different initialization, which fails to capture the sufficiently diverse features. This motivates us to use tri-training and develop the triple-view encoder to utilize the encoders with different architectures to derive diverse features, and exploit the knowledge distillation skill to learn the complementary semantics among these encoders. Moreover, existing methods simply concatenate the features from both encoder and decoder, resulting in redundant features that require large memory cost. This inspires us to devise a dual-frequency decoder that selects those important features by projecting the features from the spatial domain to the frequency domain, where the dual-frequency channel attention mechanism is introduced to model the feature importance. Therefore, we propose a Triple-view Knowledge Distillation framework, termed TriKD, for semi-supervised semantic segmentation, including the triple-view encoder and the dual-frequency decoder. Extensive experiments were conducted on two benchmarks, \ie, Pascal VOC 2012 and Cityscapes, whose results verify the superiority of the proposed method with a good tradeoff between precision and inference speed.

📄 PDF Abstract BibTeX arXiv:2309.12557

Code (0)

등록된 구현이 없습니다.

Tasks

DecoderFeature ImportanceKnowledge DistillationSemantic SegmentationSemi-Supervised Semantic Segmentation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Similar Papers 제목 키워드 기반

DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining

2023-05-20 · Weifeng Jiang, Qianren Mao, Chenghua Lin, JianXin Li 외

Many text mining models are constructed by fine-tuning a large deep pre-trained language model (PLM) in downstream tasks. However, a significant challenge nowadays is maintaining performance when we use a lightweight mod…

Extractive SummarizationKnowledge DistillationLanguage ModelingLanguage Modelling+3

Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation

2024-12-18 · Kaiwen Huang, Tao Zhou, Huazhu Fu, Yizhe Zhang 외

The limited availability of labeled data has driven advancements in semi-supervised learning for medical image segmentation. Modern large-scale models tailored for general segmentation, such as the Segment Anything Model…

Image SegmentationKnowledge DistillationMedical Image SegmentationSegmentation+2

Knowledge Distillation for Road Detection based on cross-model Semi-Supervised Learning

2024-02-07 · Wanli Ma, Oktay Karakus, Paul L. Rosin

The advancement of knowledge distillation has played a crucial role in enabling the transfer of knowledge from larger teacher models to smaller and more efficient student models, and is particularly beneficial for online…

Knowledge DistillationRoad Segmentation

Progressive Self-Distillation for Ground-to-Aerial Perception Knowledge Transfer

2022-08-29 · Junjie Hu, Chenyou Fan, Mete Ozay, Hua Feng 외

We study a practical yet hasn't been explored problem: how a drone can perceive in an environment from different flight heights. Unlike autonomous driving, where the perception is always conducted from a ground viewpoint…

Autonomous DrivingKnowledge DistillationTransfer Learning

Triple-View Feature Learning for Medical Image Segmentation

2022-08-12 · Ziyang Wang, Irina Voiculescu

Deep learning models, e.g. supervised Encoder-Decoder style networks, exhibit promising performance in medical image segmentation, but come with a high labelling cost. We propose TriSegNet, a semi-supervised semantic seg…

DecoderImage SegmentationMedical Image SegmentationSegmentation+2