Comparing Apples to Apple: The Effects of Stemmers on Topic Models
Rule-based stemmers such as the Porter stemmer are frequently used to preprocess English corpora for topic modeling. In this work, we train and evaluate topic models on a variety of corpora using several different stemming algorithms. We examine several different quantitative measures of the resulting models, including likelihood, coherence, model stability, and entropy. Despite their frequent use in topic modeling, we find that stemmers produce no meaningful improvement in likelihood and coherence and in fact can degrade topic stability.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalSemantic Textual SimilarityTopic ModelsSimilar Papers 제목 키워드 기반
Design of an Intelligent Vision Algorithm for Recognition and Classification of Apples in an Orchard Scene
Apple is one of the remarkable fresh fruit that contains a high degree of nutritious and medicinal value. Hand harvesting of apples by seasonal farmworkers increases physical damages on the surface of these fruits, which…
MarketingApples to Apples: A Systematic Evaluation of Topic Models
From statistical to neural models, a wide variety of topic modelling algorithms have been proposed in the literature. However, because of the diversity of datasets and metrics, there have not been many efforts to systema…
DiversityTopic ModelsWord EmbeddingsA New Simple Vision Algorithm for Detecting the Enzymic Browning Defects in Golden Delicious Apples
In this work, a simple vision algorithm is designed and implemented to extract and identify the surface defects on the Golden Delicious apples caused by the enzymic browning process. 34 Golden Delicious apples were selec…
Localizing Small Apples in Complex Apple Orchard Environments
The localization of fruits is an essential first step in automated agricultural pipelines for yield estimation or fruit picking. One example of this is the localization of apples in images of entire apple trees. Since th…
ObjectObject Proposal GenerationTowards Apples to Apples for AI Evaluations: From Real-World Use Cases to Evaluation Scenarios
AI measurement science has a wide variety of methodologies and measurements for comparing AI systems, resulting in what often appear to be "apples-to-oranges" comparisons across AI evaluations. To move toward "apples-to-…