A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification
We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained language model (e.g., M-BERT). It leverages the multilingual nature of Wikidata: entities in multiple languages representing the same concept are defined with a unique identifier. This enables entities described in multiple languages to be represented using shared embeddings. A model trained on entity features in a resource-rich language can thus be directly applied to other languages. Our experimental results on cross-lingual topic classification (using the MLDoc and TED-CLDC datasets) and entity typing (using the SHINRA2020-ML dataset) show that the proposed model consistently outperforms state-of-the-art models.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationEntity TypingLanguage ModelingLanguage Modellingtext-classificationText ClassificationTopic ClassificationSimilar Papers 제목 키워드 기반
A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification
We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained language model (e.g., M-BERT). It leverages t…
ClassificationEntity TypingLanguage ModelingLanguage Modelling+3LEMONADE: A Large Multilingual Expert-Annotated Abstractive Event Dataset for the Real World
This paper presents LEMONADE, a large-scale conflict event dataset comprising 39,786 events across 20 languages and 171 countries, with extensive coverage of region-specific entities. LEMONADE is based on a partially rea…
document understandingEntity LinkingEvent ExtractionZero-shot Event ExtractionCo-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment
Multilingual knowledge graph (KG) embeddings provide latent semantic representations of entities and structured knowledge with cross-lingual inferences, which benefit various knowledge-driven cross-lingual NLP tasks. How…
Entity AlignmentKnowledge GraphsThe Multilingual Curse at the Retrieval Layer: Evidence from Amharic
Multilingual retrieval increasingly underpins cross-lingual question answering and retrieval-augmented generation. Strong zero-shot scores on multilingual benchmarks are often taken as evidence that current encoders tran…
Cross-Lingual Question AnsweringPassage RetrievalCross-Lingual Text Classification with Multilingual Distillation and Zero-Shot-Aware Training
Multilingual pre-trained language models (MPLMs) not only can handle tasks in different languages but also exhibit surprising zero-shot cross-lingual transferability. However, MPLMs usually are not able to achieve compar…
Language ModelingLanguage Modellingtext-classificationText Classification