paper-with-me

Papers

Multi-Channel Multi-Domain based Knowledge Distillation Algorithm for Sleep Staging with Single-Channel EEG

2024-01-07 · Chao Zhang, Yiqiao Liao, Siqi Han, Milin Zhang, Zhihua Wang, Xiang Xie

This paper proposed a Multi-Channel Multi-Domain (MCMD) based knowledge distillation algorithm for sleep staging using single-channel EEG. Both knowledge from different domains and different channels are learnt in the proposed algorithm, simultaneously. A multi-channel pre-training and single-channel fine-tuning scheme is used in the proposed work. The knowledge from different channels in the source domain is transferred to the single-channel model in the target domain. A pre-trained teacher-student model scheme is used to distill knowledge from the multi-channel teacher model to the single-channel student model combining with output transfer and intermediate feature transfer in the target domain. The proposed algorithm achieves a state-of-the-art single-channel sleep staging accuracy of 86.5%, with only 0.6% deterioration from the state-of-the-art multi-channel model. There is an improvement of 2% compared to the baseline model. The experimental results show that knowledge from multiple domains (different datasets) and multiple channels (e.g. EMG, EOG) could be transferred to single-channel sleep staging.

📄 PDF Abstract BibTeX arXiv:2401.03430

Code (0)

등록된 구현이 없습니다.

Tasks

EEGKnowledge DistillationSleep Staging

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Similar Papers 제목 키워드 기반

Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization

2022-10-07 · Shota Horiguchi, Yuki Takashima, Shinji Watanabe, Paola Garcia

Due to the high performance of multi-channel speech processing, we can use the outputs from a multi-channel model as teacher labels when training a single-channel model with knowledge distillation. To the contrary, it is…

Knowledge Distillationspeaker-diarizationSpeaker DiarizationTransfer Learning

WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition

2022-12-07 · Jun-Yu Ma, Beiduo Chen, Jia-Chen Gu, Zhen-Hua Ling 외

Zero-shot cross-lingual named entity recognition (NER) aims at transferring knowledge from annotated and rich-resource data in source languages to unlabeled and lean-resource data in target languages. Existing mainstream…

Cross-Lingual NERDomain Adaptationnamed-entity-recognitionNamed Entity Recognition+2

Multi scale Feature Extraction and Fusion for Online Knowledge Distillation

2022-06-16 · Panpan Zou, Yinglei Teng, Tao Niu

Online knowledge distillation conducts knowledge transfer among all student models to alleviate the reliance on pre-trained models. However, existing online methods rely heavily on the prediction distributions and neglec…

Knowledge DistillationTransfer Learning

HYDRA: HYbrid knowledge Distillation and spectral Reconstruction Algorithm for high channel hyperspectral camera applications

2025-10-18 · Christopher Thirgood, Oscar Mendez, Erin Ling, Jon Storey 외 arxiv

Hyperspectral images (HSI) promise to support a range of new applications in computer vision. Recent research has explored the feasibility of generalizable Spectral Reconstruction (SR), the problem of recovering a HSI fr…

Spectral ReconstructionKnowledge Distillation

Channel Distillation: Channel-Wise Attention for Knowledge Distillation

2020-06-02 · Zaida Zhou, Chaoran Zhuge, Xinwei Guan, Wen Liu

Knowledge distillation is to transfer the knowledge from the data learned by the teacher network to the student network, so that the student has the advantage of less parameters and less calculations, and the accuracy is…

Knowledge Distillation