A New Knowledge Distillation Network for Incremental Few-Shot Surface Defect Detection
Surface defect detection is one of the most essential processes for industrial quality inspection. Deep learning-based surface defect detection methods have shown great potential. However, the well-performed models usually require large training data and can only detect defects that appeared in the training stage. When facing incremental few-shot data, defect detection models inevitably suffer from catastrophic forgetting and misclassification problem. To solve these problems, this paper proposes a new knowledge distillation network, called Dual Knowledge Align Network (DKAN). The proposed DKAN method follows a pretraining-finetuning transfer learning paradigm and a knowledge distillation framework is designed for fine-tuning. Specifically, an Incremental RCNN is proposed to achieve decoupled stable feature representation of different categories. Under this framework, a Feature Knowledge Align (FKA) loss is designed between class-agnostic feature maps to deal with catastrophic forgetting problems, and a Logit Knowledge Align (LKA) loss is deployed between logit distributions to tackle misclassification problems. Experiments have been conducted on the incremental Few-shot NEU-DET dataset and results show that DKAN outperforms other methods on various few-shot scenes, up to 6.65% on the mean Average Precision metric, which proves the effectiveness of the proposed method.
Code (1)
Tasks
Defect DetectionKnowledge DistillationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning
Incremental learning targets at achieving good performance on new categories without forgetting old ones. Knowledge distillation has been shown critical in preserving the performance on old classes. Conventional methods,…
Incremental LearningKnowledge DistillationCD^2: Constrained Dataset Distillation for Few-Shot Class-Incremental Learning
Few-shot class-incremental learning (FSCIL) receives significant attention from the public to perform classification continuously with a few training samples, which suffers from the key catastrophic forgetting problem. E…
Few-Shot Class-Incremental LearningPre-trained Vision and Language Transformers Are Few-Shot Incremental Learners
Few-Shot Class Incremental Learning (FSCIL) is a task that requires a model to learn new classes incrementally without forgetting when only a few samples for each class are given. FSCIL encounters two significant challen…
class-incremental learningClass Incremental LearningFew-Shot Class-Incremental LearningIncremental Learning+2Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition
Most meta-learning approaches assume the existence of a very large set of labeled data available for episodic meta-learning of base knowledge. This contrasts with the more realistic continual learning paradigm in which d…
Continual LearningKnowledge DistillationMeta-LearningContinual learning for surface defect segmentation by subnetwork creation and selection
We introduce a new continual (or lifelong) learning algorithm called LDA-CP&S that performs segmentation tasks without undergoing catastrophic forgetting. The method is applied to two different surface defect segmentatio…
Continual LearningLifelong learningSegmentation