PuoBERTa: Training and evaluation of a curated language model for Setswana

2023-10-13 · Vukosi Marivate, Moseli Mots'oehli, Valencia Wagner, Richard Lastrucci, Isheanesu Dzingirai

Natural language processing (NLP) has made significant progress for well-resourced languages such as English but lagged behind for low-resource languages like Setswana. This paper addresses this gap by presenting PuoBERTa, a customised masked language model trained specifically for Setswana. We cover how we collected, curated, and prepared diverse monolingual texts to generate a high-quality corpus for PuoBERTa's training. Building upon previous efforts in creating monolingual resources for Setswana, we evaluated PuoBERTa across several NLP tasks, including part-of-speech (POS) tagging, named entity recognition (NER), and news categorisation. Additionally, we introduced a new Setswana news categorisation dataset and provided the initial benchmarks using PuoBERTa. Our work demonstrates the efficacy of PuoBERTa in fostering NLP capabilities for understudied languages like Setswana and paves the way for future research directions.

📄 PDF Abstract BibTeX arXiv:2310.09141

Code (2)

dsfsi/puoberta 공식 구현

dsfsi/puodata 공식 구현

Tasks

Language ModelingLanguage Modellingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER Part-Of-Speech TaggingPOSPOS Tagging

BOTS-LM: Training Large Language Models for Setswana

2024-08-05 · Nathan Brown, Vukosi Marivate

In this work we present BOTS-LM, a series of bilingual language models proficient in both Setswana and English. Leveraging recent advancements in data availability and efficient fine-tuning, BOTS-LM achieves performance …

Computational Efficiency Language Modeling Language Modelling Large Language Model+3

Investigating an approach for low resource language dataset creation, curation and classification: Setswana and Sepedi

2020-02-18 · LREC 2020 5 · Vukosi Marivate, Tshephisho Sefara, Vongani Chabalala, Keamogetswe Makhaya 외

The recent advances in Natural Language Processing have been a boon for well-represented languages in terms of available curated data and research resources. One of the challenges for low-resourced languages is clear gui…

Classification Data Augmentation General Classification Topic Classification

Training Cross-Lingual embeddings for Setswana and Sepedi

2021-11-11 · Mack Makgatho, Vukosi Marivate, Tshephisho Sefara, Valencia Wagner

African languages still lag in the advances of Natural Language Processing techniques, one reason being the lack of representative data, having a technique that can transfer information between languages can help mitigat…

Cross-Lingual Transfer Semantic Similarity Semantic Textual Similarity Word Embeddings

Low resource language dataset creation, curation and classification: Setswana and Sepedi -- Extended Abstract

2020-03-30 · Vukosi Marivate, Tshephisho Sefara, Vongani Chabalala, Keamogetswe Makhaya 외

The recent advances in Natural Language Processing have only been a boon for well represented languages, negating research in lesser known global languages. This is in part due to the availability of curated data and res…

Data Augmentation General Classification Topic Classification

Practical Approach on Implementation of WordNets for South African Languages

2021-01-01 · EACL (GWC) 2021 1 · Tshephisho Joseph Sefara, Tumisho Billson Mokgonyane, Vukosi Marivate

This paper proposes the implementation of WordNets for five South African languages, namely, Sepedi, Setswana, Tshivenda, isiZulu and isiXhosa to be added to open multilingual WordNets (OMW) on natural language toolkit (…