Cross-Lingual Transfer for Distantly Supervised and Low-resources Indonesian NER
Manually annotated corpora for low-resource languages are usually small in quantity (gold), or large but distantly supervised (silver). Inspired by recent progress of injecting pre-trained language model (LM) on many Natural Language Processing (NLP) task, we proposed to fine-tune pre-trained language model from high-resources languages to low-resources languages to improve the performance of both scenarios. Our empirical experiment demonstrates significant improvement when fine-tuning pre-trained language model in cross-lingual transfer scenarios for small gold corpus and competitive results in large silver compare to supervised cross-lingual transfer, which will be useful when there is no parallel annotation in the same task to begin. We compare our proposed method of cross-lingual transfer using pre-trained LM to different sources of transfer such as mono-lingual LM and Part-of-Speech tagging (POS) in the downstream task of both large silver and small gold NER dataset by exploiting character-level input of bi-directional language model task.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferLanguage ModelingLanguage ModellingNERPart-Of-Speech TaggingPOSSimilar Papers 제목 키워드 기반
Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages
Fine-grained entity typing (FGET) aims to classify named entity mentions into fine-grained entity types, which is meaningful for entity-related NLP tasks. For FGET, a key challenge is the low-resource problem — the compl…
Contrastive LearningEntity TypingMachine TranslationCombining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction
Distantly Supervised Relation Extraction (DSRE) remains a long-standing challenge in NLP, where models must learn from noisy bag-level annotations while making sentence-level predictions. While existing state-of-the-art …
Relation ExtractionCan Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi
Building natural language processing systems for non standardized and low resource languages is a difficult challenge. The recent success of large-scale multilingual pretrained language models provides new modeling tools…
Dependency ParsingPart-Of-Speech TaggingTransliterationPARE: A Simple and Strong Baseline for Monolingual and Multilingual Distantly Supervised Relation Extraction
Neural models for distantly supervised relation extraction (DS-RE) encode each sentence in an entity-pair bag separately. These are then aggregated for bag-level relation prediction. Since, at encoding time, these approa…
RelationRelation ExtractionRelation PredictionSentencePARE: A Simple and Strong Baseline for Monolingual and Multilingual Distantly Supervised Relation Extraction
Neural models for distantly supervised relation extraction (DS-RE) encode each sentence in an entity-pair bag separately. These are then aggregated for bag-level relation prediction. Since, at encoding time, these approa…
RelationRelation ExtractionRelation PredictionSentence