Normalized Feature Distillation for Semantic Segmentation

2022-07-12 · Tao Liu, Xi Yang, Chenshu Chen

As a promising approach in model compression, knowledge distillation improves the performance of a compact model by transferring the knowledge from a cumbersome one. The kind of knowledge used to guide the training of the student is important. Previous distillation methods in semantic segmentation strive to extract various forms of knowledge from the features, which involve elaborate manual design relying on prior information and have limited performance gains. In this paper, we propose a simple yet effective feature distillation method called normalized feature distillation (NFD), aiming to enable effective distillation with the original features without the need to manually design new forms of knowledge. The key idea is to prevent the student from focusing on imitating the magnitude of the teacher's feature response by normalization. Our method achieves state-of-the-art distillation results for semantic segmentation on Cityscapes, VOC 2012, and ADE20K datasets. Code will be available.

📄 PDF Abstract BibTeX arXiv:2207.05256

Code (0)

등록된 구현이 없습니다.

Tasks

Knowledge Distillation Model Compression Segmentation Semantic Segmentation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Distilling Inter-Class Distance for Semantic Segmentation

2022-05-07 · Zhengbo Zhang, Chunluan Zhou, Zhigang Tu

Knowledge distillation is widely adopted in semantic segmentation to reduce the computation cost.The previous knowledge distillation methods for semantic segmentation focus on pixel-wise feature alignment and intra-class…

Knowledge Distillation Position Segmentation Semantic Segmentation

FAKD: Feature Augmented Knowledge Distillation for Semantic Segmentation

2022-08-30 · Jianlong Yuan, Qian Qi, Fei Du, Zhibin Wang 외

In this work, we explore data augmentations for knowledge distillation on semantic segmentation. To avoid over-fitting to the noise in the teacher network, a large number of training examples is essential for knowledge d…

Knowledge Distillation Segmentation Semantic Segmentation

Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Data

2021-03-25 · Zhiyuan Wu, Yu Jiang, Chupeng Cui, Zongmin Yang 외

Semantic segmentation of road scenes is one of the key technologies for realizing autonomous driving scene perception, and the effectiveness of deep Convolutional Neural Networks(CNNs) for this task has been demonstrated…

Autonomous Driving Few-Shot Learning Knowledge Distillation Real-Time Semantic Segmentation+3

CUS3D :CLIP-based Unsupervised 3D Segmentation via Object-level Denoise

2024-09-21 · Fuyang Yu, Runze Tian, Zhen Wang, Xiaochuan Wang 외

To ease the difficulty of acquiring annotation labels in 3D data, a common method is using unsupervised and open-vocabulary semantic segmentation, which leverage 2D CLIP semantic knowledge. In this paper, unlike previous…

Open Vocabulary Semantic Segmentation Open-Vocabulary Semantic Segmentation Segmentation Semantic Segmentation+1

Intra-class Feature Variation Distillation for Semantic Segmentation

2020-08-01 · ECCV 2020 8 · Yukang Wang, Wei Zhou, Tao Jiang, Xiang Bai 외

Current state-of-the-art semantic segmentation methods usually require high computational resources for accurate segmentation. One promising way to achieve a good trade-off between segmentation accuracy and efficiency is…