Comparative Analysis of Multilingual Text Classification & Identification through Deep Learning and Embedding Visualization
This research conducts a comparative study on multilingual text classification methods, utilizing deep learning and embedding visualization. The study employs LangDetect, LangId, FastText, and Sentence Transformer on a dataset encompassing 17 languages. It explores dimensionality's impact on clustering, revealing FastText's clearer clustering in 2D visualization due to its extensive multilingual corpus training. Notably, the FastText multi-layer perceptron model achieved remarkable accuracy, precision, recall, and F1 score, outperforming the Sentence Transformer model. The study underscores the effectiveness of these techniques in multilingual text classification, emphasizing the importance of large multilingual corpora for training embeddings. It lays the groundwork for future research and assists practitioners in developing language detection and classification systems. Additionally, it includes the comparison of multi-layer perceptron, LSTM, and Convolution models for classification.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationClusteringMultilingual text classificationSentencetext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
cs@DravidianLangTech-EACL2021: Offensive Language Identification Based On Multilingual BERT Model
This paper introduces the related content of the task “Offensive Language Identification in Dravidian LANGUAGES-EACL 2021”. The task requires us to classify Dravidian languages collected from social media into Not-Offens…
Language Identificationtext-classificationText ClassificationComparative Performance of Advanced NLP Models and LLMs in Multilingual Geo-Entity Detection
The integration of advanced Natural Language Processing (NLP) methodologies and Large Language Models (LLMs) has significantly enhanced the extraction and analysis of geospatial data from multilingual texts, impacting se…
ALBERTI, a Multilingual Domain Specific Language Model for Poetry Analysis
The computational analysis of poetry is limited by the scarcity of tools to automatically analyze and scan poems. In a multilingual settings, the problem is exacerbated as scansion and rhyme systems only exist for indivi…
Language ModelingLanguage ModellingLarge Language ModelMultilingual Pre-Trained Transformers and Convolutional NN Classification Models for Technical Domain Identification
In this paper, we present a transfer learning system to perform technical domain identification on multilingual text data. We have submitted two runs, one uses the transformer model BERT, and the other uses XLM-ROBERTa w…
General Classificationtext-classificationText ClassificationTransfer LearningMono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi
Transformers are the most eminent architectures used for a vast range of Natural Language Processing tasks. These models are pre-trained over a large text corpus and are meant to serve state-of-the-art results over tasks…
ArticlesClassificationHate Speech DetectionSentence+5