Comprehensive Evaluation of Large Language Models for Topic Modeling
Recent work utilizes Large Language Models (LLMs) for topic modeling, generating comprehensible topic labels for given documents. However, their performance has mainly been evaluated qualitatively, and there remains room for quantitative investigation of their capabilities. In this paper, we quantitatively evaluate LLMs from multiple perspectives: the quality of topics, the impact of LLM-specific concerns, such as hallucination and shortcuts for limited documents, and LLMs' controllability of topic categories via prompts. Our findings show that LLMs can identify coherent and diverse topics with few hallucinations but may take shortcuts by focusing only on parts of documents. We also found that their controllability is limited.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationTopic ModelsSimilar Papers 제목 키워드 기반
LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Models
Topic modeling has been a widely used tool for unsupervised text analysis. However, comprehensive evaluations of a topic model remain challenging. Existing evaluation methods are either less comparable across different m…
Language ModelingLanguage ModellingTopic ModelsTopic Modeling in Marathi
While topic modeling in English has become a prevalent and well-explored area, venturing into topic modeling for Indic languages remains relatively rare. The limited availability of resources, diverse linguistic structur…
DiversityEnhanced Short Text Modeling: Leveraging Large Language Models for Topic Refinement
Crafting effective topic models for brief texts, like tweets and news headlines, is essential for capturing the swift shifts in social dynamics. Traditional topic models, however, often fall short in accurately represent…
Prompt EngineeringTopic ModelsNeural Multimodal Topic Modeling: A Comprehensive Evaluation
Neural topic models can successfully find coherent and diverse topics in textual data. However, they are limited in dealing with multimodal datasets (e.g., images and text). This paper presents the first systematic and c…
Topic ModelsGHTM: A Graph-based Hybrid Topic Modeling Approach with a Benchmark Dataset for the Low-Resource Bengali Language
Topic modeling is a Natural Language Processing (NLP) technique used to discover latent themes and abstract topics from text corpora by grouping co-occurring keywords. Although widely researched in English, topic modelin…
Computational Efficiency