L3Cube-MahaNER: A Marathi Named Entity Recognition Dataset and BERT models
Named Entity Recognition (NER) is a basic NLP task and finds major applications in conversational and search systems. It helps us identify key entities in a sentence used for the downstream application. NER or similar slot filling systems for popular languages have been heavily used in commercial applications. In this work, we focus on Marathi, an Indian language, spoken prominently by the people of Maharashtra state. Marathi is a low resource language and still lacks useful NER resources. We present L3Cube-MahaNER, the first major gold standard named entity recognition dataset in Marathi. We also describe the manual annotation guidelines followed during the process. In the end, we benchmark the dataset on different CNN, LSTM, and Transformer based models like mBERT, XLM-RoBERTa, IndicBERT, MahaBERT, etc. The MahaBERT provides the best performance among all the models. The data and models are available at https://github.com/l3cube-pune/MarathiNLP .
Code (1)
Tasks
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERSentenceslot-fillingSlot FillingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
L3Cube-MahaNLP: Marathi Natural Language Processing Datasets, Models, and Library
Despite being the third most popular language in India, the Marathi language lacks useful NLP resources. Moreover, popular NLP libraries do not have support for the Marathi language. With L3Cube-MahaNLP, we aim to build …
Hate Speech DetectionLanguage ModelingLanguage Modellingnamed-entity-recognition+3BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathi
Named Entity Recognition (NER) for low-resource languages such as Marathi remains a challenging task due to limited annotated resources and linguistic complexity. Although recent Large Language Models (LLMs) have demonst…
On Significance of Subword tokenization for Low Resource and Efficient Named Entity Recognition: A case study in Marathi
Named Entity Recognition (NER) systems play a vital role in NLP applications such as machine translation, summarization, and question-answering. These systems identify named entities, which encompass real-world concepts …
Computational EfficiencyMachine Translationnamed-entity-recognitionNamed Entity Recognition+3L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources
We present L3Cube-MahaCorpus a Marathi monolingual data set scraped from different internet sources. We expand the existing Marathi monolingual corpus with 24.8M sentences and 289M tokens. We further present, MahaBERT, M…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+4L3Cube-MahaSocialNER: A Social Media based Marathi NER Dataset and BERT models
This work introduces the L3Cube-MahaSocialNER dataset, the first and largest social media dataset specifically designed for Named Entity Recognition (NER) in the Marathi language. The dataset comprises 18,000 manually la…
Marketingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1