Cross-Lingual Task-Specific Representation Learning for Text Classification in Resource Poor Languages
Neural network models have shown promising results for text classification. However, these solutions are limited by their dependence on the availability of annotated data. The prospect of leveraging resource-rich languages to enhance the text classification of resource-poor languages is fascinating. The performance on resource-poor languages can significantly improve if the resource availability constraints can be offset. To this end, we present a twin Bidirectional Long Short Term Memory (Bi-LSTM) network with shared parameters consolidated by a contrastive loss function (based on a similarity metric). The model learns the representation of resource-poor and resource-rich sentences in a common space by using the similarity between their assigned annotation tags. Hence, the model projects sentences with similar tags closer and those with different tags farther from each other. We evaluated our model on the classification tasks of sentiment analysis and emoji prediction for resource-poor languages - Hindi and Telugu and resource-rich languages - English and Spanish. Our model significantly outperforms the state-of-the-art approaches in both the tasks across all metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationRepresentation LearningSentiment Analysistext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model
A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual text representations that can be used to …
Few-Shot LearningLanguage ModelingLanguage ModellingSentenceA Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT
Recently, multilingual BERT works remarkably well on cross-lingual transfer tasks, superior to static non-contextualized word embeddings. In this work, we provide an in-depth experimental study to supplement the existing…
Cross-Lingual TransferTranslationWord EmbeddingsDistilling Monolingual and Crosslingual Word-in-Context Representations
In this study, we propose a method that distils representations of word meaning in context from a pre-trained masked language model in both monolingual and crosslingual settings. Word representations are the basis for co…
Language ModelingLanguage ModellingSemantic Textual SimilaritySTSUnsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model
Recent research indicates that pretraining cross-lingual language models on large-scale unlabeled texts yields significant performance improvements over various cross-lingual and low-resource tasks. Through training on o…
Domain AdaptationLanguage ModelingLanguage ModellingMutual Information Estimation+1Backretrieval: An Image-Pivoted Evaluation Metric for Cross-Lingual Text Representations Without Parallel Corpora
Cross-lingual text representations have gained popularity lately and act as the backbone of many tasks such as unsupervised machine translation and cross-lingual information retrieval, to name a few. However, evaluation …
Cross-Lingual Information RetrievalInformation RetrievalMachine TranslationRetrieval+2