Double Reverse Regularization Network Based on Self-Knowledge Distillation for SAR Object Classification
In current synthetic aperture radar (SAR) object classification, one of the major challenges is the severe overfitting issue due to the limited dataset (few-shot) and noisy data. Considering the advantages of knowledge distillation as a learned label smoothing regularization, this paper proposes a novel Double Reverse Regularization Network based on Self-Knowledge Distillation (DRRNet-SKD). Specifically, through exploring the effect of distillation weight on the process of distillation, we are inspired to adopt the double reverse thought to implement an effective regularization network by combining offline and online distillation in a complementary way. Then, the Adaptive Weight Assignment (AWA) module is designed to adaptively assign two reverse-changing weights based on the network performance, allowing the student network to better benefit from both teachers. The experimental results on OpenSARShip and FUSAR-Ship demonstrate that DRRNet-SKD exhibits remarkable performance improvement on classical CNNs, outperforming state-of-the-art self-knowledge distillation methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationSelf-Knowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self-Distilled Policy Gradient
On-policy self-distillation, where a language model conditions on privileged context to supervise its own generations, is a promising source of dense supervision for sparse-reward reinforcement learning. Actually, it can…
Reinforcement LearningLightweight Self-Knowledge Distillation with Multi-source Information Fusion
Knowledge Distillation (KD) is a powerful technique for transferring knowledge between neural network models, where a pre-trained teacher model is used to facilitate the training of the target student model. However, the…
Knowledge DistillationSelf-Knowledge DistillationSelf-Distillation Amplifies Regularization in Hilbert Space
Knowledge distillation introduced in the deep learning context is a method to transfer knowledge from one architecture to another. In particular, when the architectures are identical, this is called self-distillation. Th…
Knowledge DistillationL2 RegularizationScale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection
Unsupervised anomaly detection using deep learning has garnered significant research attention due to its broad applicability, particularly in medical imaging where labeled anomalous data are scarce. While earlier approa…
Anomaly DetectionKnowledge DistillationSelf-Supervised LearningUnsupervised Anomaly DetectionSolvable Model for Inheriting the Regularization through Knowledge Distillation
In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core properties. Knowledge distillation where a smalle…
Knowledge DistillationTransfer Learning