Short Text Clustering with Transformers

2021-01-31 · Leonid Pugachev, Mikhail Burtsev

Recent techniques for the task of short text clustering often rely on word embeddings as a transfer learning component. This paper shows that sentence vector representations from Transformers in conjunction with different clustering methods can be successfully applied to address the task. Furthermore, we demonstrate that the algorithm of enhancement of clustering via iterative classification can further improve initial clustering performance with different classifiers, including those based on pre-trained Transformer language models.

📄 PDF Abstract BibTeX arXiv:2102.00541

Code (0)

등록된 구현이 없습니다.

Tasks

ClusteringSentenceShort Text Clustering Text Clustering Transfer LearningWord Embeddings

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Attention 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Adam 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Self-supervised Document Clustering Based on BERT with Data Augment

2020-11-17 · Haoxiang Shi, Cen Wang

Contrastive learning is a promising approach to unsupervised learning, as it inherits the advantages of well-studied deep models without a dedicated and complex model design. In this paper, based on bidirectional encoder…

Clustering Contrastive Learning Data Augmentation Text Clustering

Federated Learning for Short Text Clustering

2023-11-23 · Mengling Hu, Chaochao Chen, Weiming Liu, Xinting Liao 외

Short text clustering has been popularly studied for its significance in mining valuable insights from many short texts. In this paper, we focus on the federated short text clustering (FSTC) problem, i.e., clustering sho…

Clustering Federated Learning Short Text Clustering Text Clustering

Enhancement of Short Text Clustering by Iterative Classification

2020-01-31 · Md Rashadul Hasan Rakib, Norbert Zeh, Magdalena Jankowska, Evangelos Milios

Short text clustering is a challenging task due to the lack of signal contained in such short texts. In this work, we propose iterative classification as a method to b o ost the clustering quality (e.g., accuracy) of sho…

Classification Clustering General Classification Short Text Clustering+1

Relation Clustering in Narrative Knowledge Graphs

2020-11-27 · Simone Mellace, K Vani, Alessandro Antonucci

When coping with literary texts such as novels or short stories, the extraction of structured information in the form of a knowledge graph might be hindered by the huge number of possible relations between the entities c…

Clustering Descriptive Knowledge Graphs Relation

Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets

2022-11-30 · Fabian Karl, Ansgar Scherp

Short text classification is a crucial and challenging aspect of Natural Language Processing. For this reason, there are numerous highly specialized short text classifiers. However, in recent short text research, State o…

Classification text-classification Text Classification