paper-with-me

Papers

Robust & Precise Knowledge Distillation-based Novel Context-Aware Predictor for Disease Detection in Brain and Gastrointestinal

2025-05-09 · Saif Ur Rehman Khan, Muhammad Nabeel Asim, Sebastian Vollmer, Andreas Dengel

Medical disease prediction, particularly through imaging, remains a challenging task due to the complexity and variability of medical data, including noise, ambiguity, and differing image quality. Recent deep learning models, including Knowledge Distillation (KD) methods, have shown promising results in brain tumor image identification but still face limitations in handling uncertainty and generalizing across diverse medical conditions. Traditional KD methods often rely on a context-unaware temperature parameter to soften teacher model predictions, which does not adapt effectively to varying uncertainty levels present in medical images. To address this issue, we propose a novel framework that integrates Ant Colony Optimization (ACO) for optimal teacher-student model selection and a novel context-aware predictor approach for temperature scaling. The proposed context-aware framework adjusts the temperature based on factors such as image quality, disease complexity, and teacher model confidence, allowing for more robust knowledge transfer. Additionally, ACO efficiently selects the most appropriate teacher-student model pair from a set of pre-trained models, outperforming current optimization methods by exploring a broader solution space and better handling complex, non-linear relationships within the data. The proposed framework is evaluated using three publicly available benchmark datasets, each corresponding to a distinct medical imaging task. The results demonstrate that the proposed framework significantly outperforms current state-of-the-art methods, achieving top accuracy rates: 98.01% on the MRI brain tumor (Kaggle) dataset, 92.81% on the Figshare MRI dataset, and 96.20% on the GastroNet dataset. This enhanced performance is further evidenced by the improved results, surpassing existing benchmarks of 97.24% (Kaggle), 91.43% (Figshare), and 95.00% (GastroNet).

📄 PDF Abstract BibTeX arXiv:2505.06381

Code (0)

등록된 구현이 없습니다.

Tasks

Disease PredictionKnowledge DistillationModel SelectionTransfer Learning

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…
SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

CAP: A Context-Aware Neural Predictor for NAS

2024-06-04 · Han Ji, Yuqi Feng, Yanan sun

Neural predictors are effective in boosting the time-consuming performance evaluation stage in neural architecture search (NAS), owing to their direct estimation of unseen architectures. Despite the effectiveness, traini…

Neural Architecture Search

Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection

2023-03-10 · CVPR 2023 1 · Luting Wang, Yi Liu, Penghui Du, Zihan Ding 외

Open-vocabulary object detection aims to provide object detectors trained on a fixed set of object categories with the generalizability to detect objects described by arbitrary text queries. Previous methods adopt knowle…

ObjectOpen-vocabulary object detectionOpen Vocabulary Object Detection

Context Unaware Knowledge Distillation for Image Retrieval

2022-07-19 · Bytasandram Yaswanth Reddy, Shiv Ram Dubey, Rakesh Kumar Sanodiya, Ravi Ranjan Prasad Karn

Existing data-dependent hashing methods use large backbone networks with millions of parameters and are computationally complex. Existing knowledge distillation methods use logits and other features of the deep (teacher)…

Image RetrievalKnowledge DistillationQuantizationRetrieval

RAM-NAS: Resource-aware Multiobjective Neural Architecture Search Method for Robot Vision Tasks

2025-09-25 · Shouren Mao, Minghao Qin, Wei Dong, Huajian Liu 외 arxiv

Neural architecture search (NAS) has shown great promise in automatically designing lightweight models. However, conventional approaches are insufficient in training the supernet and pay little attention to actual robot …

Neural Architecture SearchKnowledge Distillation

CLoCKDistill: Consistent Location-and-Context-aware Knowledge Distillation for DETRs

2025-02-15 · Qizhen Lan, Qing Tian

Object detection has advanced significantly with Detection Transformers (DETRs). However, these models are computationally demanding, posing challenges for deployment in resource-constrained environments (e.g., self-driv…

DenoisingKnowledge Distillationobject-detectionObject Detection+1