Cross-lingual Named Entity List Search via Transliteration
Out-of-vocabulary words are still a challenge in cross-lingual Natural Language Processing tasks, for which transliteration from source to target language or script is one of the solutions. In this study, we collect a personal name dataset in 445 Wikidata languages (37 scripts), train Transformer-based multilingual transliteration models on 6 high- and 4 less-resourced languages, compare them with bilingual models from (Merhav and Ash, 2018) and determine that multilingual models perform better for less-resourced languages. We discover that intrinsic evaluation, i.e comparison to a single gold standard, might not be appropriate in the task of transliteration due to its high variability. For this reason, we propose using extrinsic evaluation of transliteration via the cross-lingual named entity list search task (e.g. personal name search in contacts list). Our code and datasets are publicly available online.
Code (0)
등록된 구현이 없습니다.
Tasks
TransliterationSimilar Papers 제목 키워드 기반
Linguistic Resources for Entity Linking Evaluation: from Monolingual to Cross-lingual
To advance information extraction and question answering technologies toward a more realistic path, the U.S. NIST (National Institute of Standards and Technology) initiated the KBP (Knowledge Base Population) task as one…
Cross-Lingual Entity LinkingEntity LinkingKnowledge Base PopulationNamed Entity Recognition (NER)+3OpenNER 1.0: Standardized Open-Access Named Entity Recognition Datasets in 50+ Languages
We present OpenNER 1.0, a standardized collection of openly available named entity recognition (NER) datasets. OpenNER contains 34 datasets spanning 51 languages, annotated in varying named entity ontologies. We correct …
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Open Named Entity Modeling from Embedding Distribution
In this paper, we report our discovery on named entity distribution in a general word embedding space, which helps an open definition on multilingual named entity definition rather than previous closed and constraint def…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Word EmbeddingsXLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment
Cross-lingual named-entity lexica are an important resource to multilingual NLP tasks such as machine translation and cross-lingual wikification. While knowledge bases contain a large number of entities in high-resource …
Machine TranslationMultilingual NLPTranslationWord AlignmentWhat Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis
Building named entity recognition (NER) models for languages that do not have much training data is a challenging task. While recent work has shown promising results on cross-lingual transfer from high-resource languages…
Cross-Lingual NERCross-Lingual Transfernamed-entity-recognitionNamed Entity Recognition+3