Cross-Image Relational Knowledge Distillation for Semantic Segmentation

2022-04-14 · CVPR 2022 1 · Chuanguang Yang, Helong Zhou, Zhulin An, Xue Jiang, Yongjun Xu, Qian Zhang

Current Knowledge Distillation (KD) methods for semantic segmentation often guide the student to mimic the teacher's structured information generated from individual data samples. However, they ignore the global semantic relations among pixels across various images that are valuable for KD. This paper proposes a novel Cross-Image Relational KD (CIRKD), which focuses on transferring structured pixel-to-pixel and pixel-to-region relations among the whole images. The motivation is that a good teacher network could construct a well-structured feature space in terms of global pixel dependencies. CIRKD makes the student mimic better structured semantic relations from the teacher, thus improving the segmentation performance. Experimental results over Cityscapes, CamVid and Pascal VOC datasets demonstrate the effectiveness of our proposed approach against state-of-the-art distillation methods. The code is available at https://github.com/winycg/CIRKD.

📄 PDF Abstract BibTeX arXiv:2204.06986

Code (1)

winycg/cirkd 공식 구현 pytorch

Tasks

Knowledge Distillation Segmentation Semantic Segmentation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

M^2C-EvDet: Multi-Domain Multi-Order Cross-Modal Knowledge Distillation for Event-based Object Detection

2026-06-23 · Wei Bao, Siqi Li, Shouan Pan, Yi Xie 외 arxiv

Event-based object Detection (EvDet), as a biologically inspired visual perception paradigm, demonstrates superior performance in scenarios demanding high temporal resolution and a wide dynamic range. Nevertheless, the i…

Knowledge Distillation Object Detection

CLIP-RD: Relative Distillation for Efficient CLIP Knowledge Distillation

2026-03-26 · Jeannie Chung, Hanna Jang, Ingyeong Yang, Uiwon Hwang 외 arxiv

CLIP aligns image and text embeddings via contrastive learning and demonstrates strong zero-shot generalization. Its large-scale architecture requires substantial computational and memory resources, motivating the distil…

Zero-shot Generalization Knowledge Distillation Contrastive Learning

Jointly Learning Knowledge Embedding and Neighborhood Consensus with Relational Knowledge Distillation for Entity Alignment

2022-01-25 · Xinhang Li, Yong Zhang, Chunxiao Xing

Entity alignment aims at integrating heterogeneous knowledge from different knowledge graphs. Recent studies employ embedding-based methods by first learning the representation of Knowledge Graphs and then performing ent…

Benchmarking Entity Alignment Entity Embeddings Knowledge Distillation+2

Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation

2024-09-04 · Kangkai Zhang, Shiming Ge, Ruixin Shi, Dan Zeng

Recognizing objects in low-resolution images is a challenging task due to the lack of informative details. Recent studies have shown that knowledge distillation approaches can effectively transfer knowledge from a high-r…

Face Recognition Knowledge Distillation Object Recognition Transfer Learning

Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation

2026-09-15 · Meisen Wang, Zhiqiang Tian, Wei Bao, Chengjie Wang 외 arxiv

Event cameras have shown great potential for robust visual perception, yet scaling event representation learning remains challenging due to the scarcity of large-scale annotated event data. Pretrained image models provid…

Representation Learning