False-Friend Detection and Entity Matching via Unsupervised Transliteration
Transliterations play an important role in multilingual entity reference resolution, because proper names increasingly travel between languages in news and social media. Previous work associated with machine translation targets transliteration only single between language pairs, focuses on specific classes of entities (such as cities and celebrities) and relies on manual curation, which limits the expression power of transliteration in multilingual environment. By contrast, we present an unsupervised transliteration model covering 69 major languages that can generate good transliterations for arbitrary strings between any language pair. Our model yields top-(1, 20, 100) averages of (32.85%, 60.44%, 83.20%) in matching gold standard transliteration compared to results from a recently-published system of (26.71%, 50.27%, 72.79%). We also show the quality of our model in detecting true and false friends from Wikipedia high frequency lexicons. Our method indicates a strong signal of pronunciation similarity and boosts the probability of finding true friends in 68 out of 69 languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationTransliterationSimilar Papers 제목 키워드 기반
Contextual Unsupervised Outlier Detection in Sequences
This work proposes an unsupervised learning framework for trajectory (sequence) outlier detection that combines ranking tests with user sequence models. The overall framework identifies sequence outliers at a desired fal…
Outlier DetectionUnsupervised False Friend Disambiguation Using Contextual Word Clusters and Parallel Word Alignments
Automatically Building a Multilingual Lexicon of False Friends With No Supervision
Cognate words, defined as words in different languages which derive from a common etymon, can be useful for language learners, who can leverage the orthographical similarity of cognates to more easily understand a text i…
Cross-Lingual Word EmbeddingsLanguage AcquisitionWord EmbeddingsTransClean: Finding False Positives in Multi-Source Entity Matching under Real-World Conditions via Transitive Consistency
We present TransClean, a method for detecting false positive predictions of entity matching algorithms under real-world conditions characterized by large-scale, noisy, and unlabeled multi-source datasets that undergo dis…
MultiEM: Efficient and Effective Unsupervised Multi-Table Entity Matching
Entity Matching (EM), which aims to identify all entity pairs referring to the same real-world entity from relational tables, is one of the most important tasks in real-world data management systems. Due to the labeling …
Management