paper-with-me

Papers

ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics

2023-02-03 · Hamed Rahimi, Hubert Naacke, Camelia Constantin, Bernd Amann

This paper presents an algorithmic family of dynamic topic models called Aligned Neural Topic Models (ANTM), which combine novel data mining algorithms to provide a modular framework for discovering evolving topics. ANTM maintains the temporal continuity of evolving topics by extracting time-aware features from documents using advanced pre-trained Large Language Models (LLMs) and employing an overlapping sliding window algorithm for sequential document clustering. This overlapping sliding window algorithm identifies a different number of topics within each time frame and aligns semantically similar document clusters across time periods. This process captures emerging and fading trends across different periods and allows for a more interpretable representation of evolving topics. Experiments on four distinct datasets show that ANTM outperforms probabilistic dynamic topic models in terms of topic coherence and diversity metrics. Moreover, it improves the scalability and flexibility of dynamic topic models by being accessible and adaptable to different types of algorithms. Additionally, a Python package is developed for researchers and scientists who wish to study the trends and evolving patterns of topics in large-scale textual data.

📄 PDF Abstract BibTeX arXiv:2302.01501

Code (1)

hamedr96/antm 공식 구현

Tasks

DiversityDynamic Topic ModelingTopic Models

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Residual Connection 설명 없음
Weight Decay 설명 없음
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.
WordPiece 설명 없음
Adam 설명 없음
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Similar Papers 제목 키워드 기반

Classification-Aware Neural Topic Model Combined With Interpretable Analysis -- For Conflict Classification

2023-08-29 · Tianyu Liang, Yida Mu, Soonho Kim, Darline Larissa Kengne Kuate 외

A large number of conflict events are affecting the world all the time. In order to analyse such conflict events effectively, this paper presents a Classification-Aware Neural Topic Model (CANTM-IA) for Conflict Informat…

Classification

Recurrent Coupled Topic Modeling over Sequential Documents

2021-06-23 · Jinjin Guo, Longbing Cao, Zhiguo Gong

The abundant sequential documents such as online archival, social media and news feeds are streamingly updated, where each chunk of documents is incorporated with smoothly evolving yet dependent topics. Such digital text…

Data AugmentationDynamic Topic Modeling

Digital Health Discussion Through Articles Published Until the Year 2021: A Digital Topic Modeling Approach

2023-07-14 · Junhyoun Sung, Hyungsook Kim

The digital health industry has grown in popularity since the 2010s, but there has been limited analysis of the topics discussed in the field across academic disciplines. This study aims to analyze the research trends of…

ArticlesElectrical Engineering

TILM: Neural Language Models with Evolving Topical Influence

2019-11-01 · CONLL 2019 11 · Shubhra Kanti Karmaker Santu, Kalyan Veeramachaneni, ChengXiang Zhai

Content of text data are often influenced by contextual factors which often evolve over time (e.g., content of social media are often influenced by topics covered in the major news streams). Existing language models do n…

Language ModelingLanguage Modelling

A Constrained Coupled Matrix-Tensor Factorization for Learning Time-evolving and Emerging Topics

2018-06-30 · Sanaz Bahargam, Evangelos E. Papalexakis

Topic discovery has witnessed a significant growth as a field of data mining at large. In particular, time-evolving topic discovery, where the evolution of a topic is taken into account has been instrumental in understan…