Continuous Learning for Children's ASR: Overcoming Catastrophic Forgetting with Elastic Weight Consolidation and Synaptic Intelligence
In this work, we present the first study addressing automatic speech recognition (ASR) for children in an online learning setting. This is particularly important for both child-centric applications and the privacy protection of minors, where training models with sequentially arriving data is critical. The conventional approach of model fine-tuning often suffers from catastrophic forgetting. To tackle this issue, we explore two established techniques: elastic weight consolidation (EWC) and synaptic intelligence (SI). Using a custom protocol on the MyST corpus, tailored to the online learning setting, we achieve relative word error rate (WER) reductions of 5.21% with EWC and 4.36% with SI, compared to the fine-tuning baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Elastic Weight Consolidation (EWC): Nuts and Bolts
In this report, we present a theoretical support of the continual learning method \textbf{Elastic Weight Consolidation}, introduced in paper titled `Overcoming catastrophic forgetting in neural networks'. Being one of th…
Continual LearningOvercoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation
Traditional object detectors are ill-equipped for incremental learning. However, fine-tuning directly on a well-trained detection model with only new data will lead to catastrophic forgetting. Knowledge distillation is a…
Class-Incremental Object DetectionIncremental LearningKnowledge Distillationobject-detection+1Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation
Continued training is an effective method for domain adaptation in neural machine translation. However, in-domain gains from adaptation come at the expense of general-domain performance. In this work, we interpret the dr…
BIG-bench Machine LearningDomain AdaptationMachine TranslationTranslationTowards continuous learning for glioma segmentation with elastic weight consolidation
When finetuning a convolutional neural network (CNN) on data from a new domain, catastrophic forgetting will reduce performance on the original training data. Elastic Weight Consolidation (EWC) is a recent technique to p…
Provable Continual Learning via Sketched Jacobian Approximations
An important problem in machine learning is the ability to learn tasks in a sequential manner. If trained with standard first-order methods most models forget previously learned tasks when trained on a new task, which is…
Continual Learning