Simplifying Multilingual News Clustering Through Projection From a Shared Space
The task of organizing and clustering multilingual news articles for media monitoring is essential to follow news stories in real time. Most approaches to this task focus on high-resource languages (mostly English), with low-resource languages being disregarded. With that in mind, we present a much simpler online system that is able to cluster an incoming stream of documents without depending on language-specific features. We empirically demonstrate that the use of multilingual contextual embeddings as the document representation significantly improves clustering quality. We challenge previous crosslingual approaches by removing the precondition of building monolingual clusters. We model the clustering process as a set of linear classifiers to aggregate similar documents, and correct closely-related multilingual clusters through merging in an online fashion. Our system achieves state-of-the-art results on a multilingual news stream clustering dataset, and we introduce a new evaluation for zero-shot news clustering in multiple languages. We make our code available as open-source.
Code (0)
등록된 구현이 없습니다.
Tasks
ArticlesClusteringSimilar Papers 제목 키워드 기반
Multilingual Clustering of Streaming News
Clustering news across languages enables efficient media monitoring by aggregating articles from multilingual sources into coherent stories. Doing so in an online setting allows scalable processing of massive news stream…
ArticlesClusteringTesting pre-trained Transformer models for Lithuanian news clustering
A recent introduction of Transformer deep learning architecture made breakthroughs in various natural language processing tasks. However, non-English languages could not leverage such new opportunities with the English t…
ClusteringXLM-RHierarchical Level-Wise News Article Clustering via Multilingual Matryoshka Embeddings
Contextual large language model embeddings are increasingly utilized for topic modeling and clustering. However, current methods often scale poorly, rely on opaque similarity metrics, and struggle in multilingual setting…
ArticlesClusteringLanguage ModelingLanguage Modelling+1Unveiling the Potential of BERTopic for Multilingual Fake News Analysis -- Use Case: Covid-19
Topic modeling is frequently being used for analysing large text corpora such as news articles or social media data. BERTopic, consisting of sentence embedding, dimension reduction, clustering, and topic extraction, is t…
ArticlesClusteringDimensionality ReductionSentence+2Batch Clustering for Multilingual News Streaming
Nowadays, digital news articles are widely available, published by various editors and often written in different languages. This large volume of diverse and unorganized information makes human reading very difficult or …
ArticlesClustering