paper-with-me

홈 › Papers

MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models

2020-08-06 · Thanh Nguyen-Duc, He Zhao, Jianfei Cai, Dinh Phung

Deep learning methods usually require a large amount of training data and lack interpretability. In this paper, we propose a novel knowledge distillation and model interpretation framework for medical image classification that jointly solves the above two issues. Specifically, to address the data-hungry issue, a small student model is learned with less data by distilling knowledge from a cumbersome pretrained teacher model. To interpret the teacher model and assist the learning of the student, an explainer module is introduced to highlight the regions of an input that are important for the predictions of the teacher model. Furthermore, the joint framework is trained by a principled way derived from the information-theoretic perspective. Our framework outperforms on the knowledge distillation and model interpretation tasks compared to state-of-the-art methods on a fundus dataset.

📄 PDF Abstract BibTeX arXiv:2008.02593

Code (0)

등록된 구현이 없습니다.

Tasks

image-classificationImage ClassificationKnowledge DistillationMedical Image Classification

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Similar Papers 제목 키워드 기반

Transferring Visual Explainability of Self-Explaining Models through Task Arithmetic

2025-07-06 · Yuya Yoshikawa, Ryotaro Shimizu, Takahiro Kawashima, Yuki Saito

In scenarios requiring both prediction and explanation efficiency for image classification, self-explaining models that perform both tasks in a single inference are effective. However, their training incurs substantial l…

image-classificationImage ClassificationTask Arithmetic

Temporal Order Preserved Optimal Transport-based Cross-modal Knowledge Transfer Learning for ASR

2024-09-03 · Xugang Lu, Peng Shen, Yu Tsao, Hisashi Kawai

Transferring linguistic knowledge from a pretrained language model (PLM) to an acoustic model has been shown to greatly improve the performance of automatic speech recognition (ASR). However, due to the heterogeneous fea…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)cross-modal alignmentspeech-recognition+2

Transferring GANs: generating images from limited data

2018-05-04 · ECCV 2018 9 · Yaxing Wang, Chenshen Wu, Luis Herranz, Joost Van de Weijer 외

Transferring the knowledge of pretrained networks to new domains by means of finetuning is a widely used practice for applications based on discriminative models. To the best of our knowledge this practice has not been s…

10-shot image generationDiversityDomain AdaptationImage Generation

Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models?

2023-11-07 · Shaoyang Xu, Junzhuo Li, Deyi Xiong

Multilingual pretrained language models serve as repositories of multilingual factual knowledge. Nevertheless, a substantial performance gap of factual knowledge probing exists between high-resource languages and low-res…

Knowledge ProbingRetrievalTransfer Learning

Multiple Representation Transfer from Large Language Models to End-to-End ASR Systems

2023-09-07 · Takuma Udagawa, Masayuki Suzuki, Gakuto Kurata, Masayasu Muraoka 외

Transferring the knowledge of large language models (LLMs) is a promising technique to incorporate linguistic knowledge into end-to-end automatic speech recognition (ASR) systems. However, existing works only transfer a …

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition