Continual Learning Long Short Term Memory
Catastrophic forgetting in neural networks indicates the performance decreasing of deep learning models on previous tasks while learning new tasks. To address this problem, we propose a novel Continual Learning Long Short Term Memory (CL-LSTM) cell in Recurrent Neural Network (RNN) in this paper. CL-LSTM considers not only the state of each individual task{'}s output gates but also the correlation of the states between tasks, so that the deep learning models can incrementally learn new tasks without catastrophically forgetting previously tasks. Experimental results demonstrate significant improvements of CL-LSTM over state-of-the-art approaches on spoken language understanding (SLU) tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningDeep LearningSpoken Language UnderstandingSimilar Papers 제목 키워드 기반
Continual Learning in Deep Neural Network by Using a Kalman Optimiser
Learning and adapting to new distributions or learning new tasks sequentially without forgetting the previously learned knowledge is a challenging phenomenon in continual learning models. Most of the conventional deep le…
Continual LearningMemVerse: Multimodal Memory for Lifelong Learning Agents
Despite rapid progress in large-scale language and vision models, AI agents still suffer from a fundamental limitation: they cannot remember. Without reliable memory, agents catastrophically forget past experiences, stru…
Multimodal ReasoningContinual LearningKnowledge GraphsViLCo-Bench: VIdeo Language COntinual learning Benchmark
Video language continual learning involves continuously adapting to information from video and text inputs, enhancing a model's ability to handle new tasks while retaining prior knowledge. This field is a relatively unde…
Continual LearningSelf-Supervised LearningSchematic Memory Persistence and Transience for Efficient and Robust Continual Learning
Continual learning is considered a promising step towards next-generation Artificial Intelligence (AI), where deep neural networks (DNNs) make decisions by continuously learning a sequence of different tasks akin to huma…
Continual LearningComplementary Learning System Empowers Online Continual Learning of Vehicle Motion Forecasting in Smart Cities
Artificial intelligence underpins most smart city services, yet deep neural network (DNN) that forecasts vehicle motion still struggle with catastrophic forgetting, the loss of earlier knowledge when models are updated. …
Continual LearningMotion Forecasting