Improved Knowledge Distillation for Land-Use Image Classification
In the present article, an improved Knowledge Distillation (KD) framework has been proposed for efficient compression of deep convolutional neural networks for land-use image classification task. Motivated by the need to achieve competitive classification accuracy while reducing computational complexity, a teacher-student learning paradigm is adopted in which a VGG16 network transfers knowledge to a lightweight MobileNetV2 model. The proposed framework integrates hard supervision from ground truth labels with a soft supervision strategy that combines Kullback-Leibler divergence and Cosine Similarity losses. Experiments conducted on three land-use datasets show that the proposed KD-based method yields improved performance, and achieves an accuracy of 99.04%, outperforming both baseline student training and single-loss distillation approaches, while retaining substantial model compression.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationImage ClassificationModel CompressionSimilar Papers 제목 키워드 기반
FedUKD: Federated UNet Model with Knowledge Distillation for Land Use Classification from Satellite and Street Views
Federated Deep Learning frameworks can be used strategically to monitor Land Use locally and infer environmental impacts globally. Distributed data from across the world would be needed to build a global model for Land U…
Knowledge DistillationModel CompressionSemantic SegmentationAdaptive Explicit Knowledge Transfer for Knowledge Distillation
Logit-based knowledge distillation (KD) for classification is cost-efficient compared to feature-based KD but often subject to inferior performance. Recently, it was shown that the performance of logit-based KD can be im…
Knowledge DistillationTransfer LearningClassification of Diabetic Retinopathy Using Unlabeled Data and Knowledge Distillation
Knowledge distillation allows transferring knowledge from a pre-trained model to another. However, it suffers from limitations, and constraints related to the two models need to be architecturally similar. Knowledge dist…
ClassificationGeneral ClassificationKnowledge DistillationMedical Image Analysis+1Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
Knowledge distillation (KD) is a technique for transferring knowledge from complex teacher models to simpler student models, significantly enhancing model efficiency and accuracy. It has demonstrated substantial advancem…
image-classificationImage ClassificationKnowledge DistillationLanguage Modeling+7Variational Knowledge Distillation for Disease Classification in Chest X-Rays
Disease classification relying solely on imaging data attracts great interest in medical image analysis. Current models could be further improved, however, by also employing Electronic Health Records (EHRs), which contai…
ClassificationGeneral Classificationimage-classificationImage Classification+3