SlovakBERT: Slovak Masked Language Model
We introduce a new Slovak masked language model called SlovakBERT. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.
Code (1)
Tasks
Language ModelingLanguage ModellingmodelPart-Of-Speech TaggingSemantic Textual SimilaritySentiment AnalysisSimilar Papers 제목 키워드 기반
Intent Classification for Bank Chatbots through LLM Fine-Tuning
This study evaluates the application of large language models (LLMs) for intent classification within a chatbot with predetermined responses designed for banking industry websites. Specifically, the research examines the…
Chatbotintent-classificationIntent ClassificationApproaches to Semantic Textual Similarity in Slovak Language: From Algorithms to Transformers
Semantic textual similarity (STS) plays a crucial role in many natural language processing tasks. While extensively studied in high-resource languages, STS remains challenging for under-resourced languages such as Slovak…
Semantic Textual SimilaritySlovKE: A Large-Scale Dataset and LLM Evaluation for Slovak Keyphrase Extraction
Keyphrase extraction for morphologically rich, low-resource languages remains understudied, largely due to the scarcity of suitable evaluation datasets. We address this gap for Slovak by constructing a dataset of 227,432…
Keyphrase ExtractionNamed Entity Recognition for Address Extraction in Speech-to-Text Transcriptions Using Synthetic Data
This paper introduces an approach for building a Named Entity Recognition (NER) model built upon a Bidirectional Encoder Representations from Transformers (BERT) architecture, specifically utilizing the SlovakBERT model.…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Measuring Gender Bias in West Slavic Language Models
Pre-trained language models have been known to perpetuate biases from the underlying datasets to downstream tasks. However, these findings are predominantly based on monolingual language models for English, whereas there…
Language Modelling