Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification
Text classification must sometimes be applied in a low-resource language with no labeled training data. However, training data may be available in a related language. We investigate whether character-level knowledge transfer from a related language helps text classification. We present a cross-lingual document classification framework (CACO) that exploits cross-lingual subword similarity by jointly training a character-based embedder and a word-based classifier. The embedder derives vector representations for input words from their written forms, and the classifier makes predictions based on the word vectors. We use a joint character representation for both the source language and the target language, which allows the embedder to generalize knowledge about source language words to target language words with similar forms. We propose a multi-task objective that can further improve the model if additional cross-lingual or monolingual resources are available. Experiments confirm that character-level knowledge transfer is more data-efficient than word-level transfer between related languages.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationCross-Lingual Document ClassificationDocument ClassificationGeneral Classificationtext-classificationText ClassificationTransfer LearningSimilar Papers 제목 키워드 기반
Subword Mapping and Anchoring across Languages
State-of-the-art multilingual systems rely on shared vocabularies that sufficiently cover all considered languages. To this end, a simple and frequently used approach makes use of subword vocabularies constructed jointly…
Cross-Lingual Natural Language InferenceMachine TranslationNatural Language InferenceTranslationExploiting Cross-Lingual Knowledge in Unsupervised Acoustic Modeling for Low-Resource Languages
(Short version of Abstract) This thesis describes an investigation on unsupervised acoustic modeling (UAM) for automatic speech recognition (ASR) in the zero-resource scenario, where only untranscribed speech data is ass…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language Acquisitionspeech-recognition+1Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations
Much work in Natural Language Processing (NLP) has been for resource-rich languages, making generalization to new, less-resourced languages challenging. We present two approaches for improving generalization to low-resou…
AvgMachine Translationnamed-entity-recognitionNamed Entity Recognition+5Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models
Human understanding of language is robust to different word choices as far as they represent similar semantic concepts. To what extent does our human intuition transfer to language models, which represent all subwords as…
A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation
Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. This paper studies the role of subword segmentation in cross-lingual transfer. We systemat…
Cross-Lingual TransferMachine Translation