Continual Neural Topic Model
In continual learning, our aim is to learn a new task without forgetting what was learned previously. In topic models, this translates to learning new topic models without forgetting previously learned topics. Previous work either considered Dynamic Topic Models (DTMs), which learn the evolution of topics based on the entire training corpus at once, or Online Topic Models, which are updated continuously based on new data but do not have long-term memory. To fill this gap, we propose the Continual Neural Topic Model (CoNTM), which continuously learns topic models at subsequent time steps without forgetting what was previously learned. This is achieved using a global prior distribution that is continuously updated. In our experiments, CoNTM consistently outperformed the dynamic topic model in terms of topic quality and predictive perplexity while being able to capture topic changes online. The analysis reveals that CoNTM can learn more diverse topics and better capture temporal changes than existing methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningTopic ModelsSimilar Papers 제목 키워드 기반
Advancing continual lifelong learning in neural information retrieval: definition, dataset, framework, and empirical evaluation
Continual learning refers to the capability of a machine learning model to learn and adapt to new information, without compromising its performance on previously learned tasks. Although several studies have investigated …
Continual LearningData AugmentationInformation RetrievalLifelong learning+1Neural Topic Modeling with Continual Lifelong Learning
Lifelong learning has recently attracted attention in building machine learning systems that continually accumulate and transfer knowledge to help future learning. Unsupervised topic modeling has been popularly used to d…
Data AugmentationInformation RetrievalLifelong learningRetrieval+1Continual Learning and Catastrophic Forgetting
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, i…
Continual LearningRenate: A Library for Real-World Continual Learning
Continual learning enables the incremental training of machine learning models on non-stationary data streams.While academic interest in the topic is high, there is little indication of the use of state-of-the-art contin…
Continual LearningEx-Model: Continual Learning from a Stream of Trained Models
Learning continually from non-stationary data streams is a challenging research topic of growing popularity in the last few years. Being able to learn, adapt, and generalize continually in an efficient, effective, and sc…
Continual Learning