paper-with-me

홈 › Papers

Improved Knowledge Distillation for Land-Use Image Classification

2026-06-12 · Arundhuti Sur, Abhiroop Chatterjee, Susmita Ghosh, Emmett Ientilucci arxiv

In the present article, an improved Knowledge Distillation (KD) framework has been proposed for efficient compression of deep convolutional neural networks for land-use image classification task. Motivated by the need to achieve competitive classification accuracy while reducing computational complexity, a teacher-student learning paradigm is adopted in which a VGG16 network transfers knowledge to a lightweight MobileNetV2 model. The proposed framework integrates hard supervision from ground truth labels with a soft supervision strategy that combines Kullback-Leibler divergence and Cosine Similarity losses. Experiments conducted on three land-use datasets show that the proposed KD-based method yields improved performance, and achieves an accuracy of 99.04%, outperforming both baseline student training and single-loss distillation approaches, while retaining substantial model compression.

📄 PDF Abstract BibTeX arXiv:2606.14886

Code (0)

등록된 구현이 없습니다.

Tasks

Knowledge DistillationImage ClassificationModel Compression

Similar Papers 제목 키워드 기반

FedUKD: Federated UNet Model with Knowledge Distillation for Land Use Classification from Satellite and Street Views

2022-12-05 · Renuga Kanagavelu, Kinshuk Dua, Pratik Garai, Susan Elias 외

Federated Deep Learning frameworks can be used strategically to monitor Land Use locally and infer environmental impacts globally. Distributed data from across the world would be needed to build a global model for Land U…

Knowledge DistillationModel CompressionSemantic Segmentation

Adaptive Explicit Knowledge Transfer for Knowledge Distillation

2024-09-03 · Hyungkeun Park, Jong-Seok Lee

Logit-based knowledge distillation (KD) for classification is cost-efficient compared to feature-based KD but often subject to inferior performance. Recently, it was shown that the performance of logit-based KD can be im…

Knowledge DistillationTransfer Learning

Classification of Diabetic Retinopathy Using Unlabeled Data and Knowledge Distillation

2020-09-01 · Sajjad Abbasi, Mohsen Hajabdollahi, Pejman Khadivi, Nader Karimi 외

Knowledge distillation allows transferring knowledge from a pre-trained model to another. However, it suffers from limitations, and constraints related to the two models need to be architecturally similar. Knowledge dist…

ClassificationGeneral ClassificationKnowledge DistillationMedical Image Analysis+1

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models

2025-04-18 · Junjie Yang, Junhao Song, Xudong Han, Ziqian Bi 외

Knowledge distillation (KD) is a technique for transferring knowledge from complex teacher models to simpler student models, significantly enhancing model efficiency and accuracy. It has demonstrated substantial advancem…

image-classificationImage ClassificationKnowledge DistillationLanguage Modeling+7

Variational Knowledge Distillation for Disease Classification in Chest X-Rays

2021-03-19 · Tom van Sonsbeek, XianTong Zhen, Marcel Worring, Ling Shao

Disease classification relying solely on imaging data attracts great interest in medical image analysis. Current models could be further improved, however, by also employing Electronic Health Records (EHRs), which contai…

ClassificationGeneral Classificationimage-classificationImage Classification+3