Ensemble Distribution Distillation for Self-Supervised Human Activity Recognition
Human Activity Recognition (HAR) has seen significant advancements with the adoption of deep learning techniques, yet challenges remain in terms of data requirements, reliability and robustness. This paper explores a novel application of Ensemble Distribution Distillation (EDD) within a self-supervised learning framework for HAR aimed at overcoming these challenges. By leveraging unlabeled data and a partially supervised training strategy, our approach yields an increase in predictive accuracy, robust estimates of uncertainty, and substantial increases in robustness against adversarial perturbation; thereby significantly improving reliability in real-world scenarios without increasing computational complexity at inference. We demonstrate this with an evaluation on several publicly available datasets. The contributions of this work include the development of a self-supervised EDD framework, an innovative data augmentation technique designed for HAR, and empirical validation of the proposed method's effectiveness in increasing robustness and reliability.
Code (0)
등록된 구현이 없습니다.
Tasks
Human Activity RecognitionSelf-Supervised LearningData AugmentationSimilar Papers 제목 키워드 기반
Self-Distribution Distillation: Efficient Uncertainty Estimation
Deep learning is increasingly being applied in safety-critical domains. For these scenarios it is important to know the level of uncertainty in a model's prediction to ensure appropriate decisions are made by the system.…
Out-of-Distribution DetectionData Distillation: Towards Omni-Supervised Learning
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lo…
Keypoint Detectionobject-detectionObject DetectionEnsemble knowledge distillation of self-supervised speech models
Distilled self-supervised models have shown competitive performance and efficiency in recent years. However, there is a lack of experience in jointly distilling multiple self-supervised speech models. In our work, we per…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Emotion RecognitionKnowledge Distillation+4EDF: Ensemble, Distill, and Fuse for Easy Video Labeling
We present a way to rapidly bootstrap object detection on unseen videos using minimal human annotations. We accomplish this by combining two complementary sources of knowledge (one generic and the other specific) using b…
Data AugmentationObjectobject-detectionObject DetectionSemi-Supervised Learning with Online Knowledge Distillation for Skin Lesion Classification
Deep Learning has emerged as a promising approach for skin lesion analysis. However, existing methods mostly rely on fully supervised learning, requiring extensive labeled data, which is challenging and costly to obtain.…
Skin Lesion ClassificationKnowledge DistillationEnsemble Learning