KDCTime: Knowledge Distillation with Calibration on InceptionTime for Time-series Classification
Time-series classification approaches based on deep neural networks are easy to be overfitting on UCR datasets, which is caused by the few-shot problem of those datasets. Therefore, in order to alleviate the overfitting phenomenon for further improving the accuracy, we first propose Label Smoothing for InceptionTime (LSTime), which adopts the information of soft labels compared to just hard labels. Next, instead of manually adjusting soft labels by LSTime, Knowledge Distillation for InceptionTime (KDTime) is proposed in order to automatically generate soft labels by the teacher model. At last, in order to rectify the incorrect predicted soft labels from the teacher model, Knowledge Distillation with Calibration for InceptionTime (KDCTime) is proposed, where it contains two optional calibrating strategies, i.e. KDC by Translating (KDCT) and KDC by Reordering (KDCR). The experimental results show that the accuracy of KDCTime is promising, while its inference time is two orders of magnitude faster than ROCKET with an acceptable training time overhead.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationTime SeriesTime Series AnalysisTime Series ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Time Series Classification for Detecting Parkinson's Disease from Wrist Motions
Parkinson's disease (PD) is a neurodegenerative condition characterized by frequently changing motor symptoms, necessitating continuous symptom monitoring for more targeted treatment. Classical time series classification…
Time SeriesTime Series ClassificationInceptionTime vs. Wavelet -- A comparison for time series classification
Neural networks were used to classify infrasound data. Two different approaches were compared. One based on the direct classification of time series data, using a custom implementation of the InceptionTime network. For t…
ClassificationTime SeriesTime Series ClassificationHard Gate Knowledge Distillation -- Leverage Calibration for Robust and Reliable Language Model
In knowledge distillation, a student model is trained with supervisions from both knowledge from a teacher and observations drawn from a training data distribution. Knowledge of a teacher is considered a subject that hol…
Knowledge DistillationLanguage ModelingLanguage ModellingSentence+1Teacher Network Calibration Improves Cross-Quality Knowledge Distillation
We investigate cross-quality knowledge distillation (CQKD), a knowledge distillation method where knowledge from a teacher network trained with full-resolution images is transferred to a student network that takes as inp…
image-classificationImage ClassificationKnowledge DistillationInceptionTime: Finding AlexNet for Time Series Classification
This paper brings deep learning at the forefront of research into Time Series Classification (TSC). TSC is the area of machine learning tasked with the categorization (or labelling) of time series. The last few decades o…
Deep LearningGeneral ClassificationTime SeriesTime Series Analysis+1