Multilingual Model Using Cross-Task Embedding Projection

2019-11-01 · CONLL 2019 11 · Jin Sakuma, Naoki Yoshinaga

We present a method for applying a neural network trained on one (resource-rich) language for a given task to other (resource-poor) languages. We accomplish this by inducing a mapping from pre-trained cross-lingual word embeddings to the embedding layer of the neural network trained on the resource-rich language. To perform element-wise cross-task embedding projection, we invent locally linear mapping which assumes and preserves the local topology across the semantic spaces before and after the projection. Experimental results on topic classification task and sentiment analysis task showed that the fully task-specific multilingual model obtained using our method outperformed the existing multilingual models with embedding layers fixed to pre-trained cross-lingual word embeddings.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Cross-Lingual Word EmbeddingsmodelSentiment AnalysisTopic ClassificationWord Embeddings

On the Language Neutrality of Pre-trained Multilingual Representations

2020-04-09 · Findings of the Association for Computational Linguistics 2020 · Jindřich Libovický, Rudolf Rosa, Alexander Fraser

Multilingual contextual embeddings, such as multilingual BERT and XLM-RoBERTa, have proved useful for many multi-lingual tasks. Previous work probed the cross-linguality of the representations indirectly using zero-shot …

Language Identification Transfer Learning Word Alignment

Harmonic Token Projection (HTP): A Vocabulary-Free, Training-Free, Deterministic, and Reversible Embedding Methodology

2025-11-10 · Tcharlies Schmitz arxiv

This paper introduces the Harmonic Token Projection (HTP), a reversible and deterministic framework for generating text embeddings without training, vocabularies, or stochastic parameters. Unlike neural embeddings that r…

Semantic Textual Similarity Semantic Similarity

FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

2026-04-20 · Santosh Kesiraju, Bolaji Yusuf, Šimon Sedláček, Oldřich Plchot 외 arxiv

This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover the lexical content from multilingual (LaBSE), multimodal (SONAR) and…

Improving Neural Knowledge Base Completion with Cross-Lingual Projections

2017-04-01 · EACL 2017 4 · Patrick Klein, Simone Paolo Ponzetto, Goran Glava{\v{s}}

In this paper we present a cross-lingual extension of a neural tensor network model for knowledge base completion. We exploit multilingual synsets from BabelNet to translate English triples to other languages and then au…

Knowledge Base Completion Link Prediction Reading Comprehension Word Embeddings

Concept Space Alignment in Multilingual LLMs

2024-10-01 · Qiwei Peng, Anders Søgaard

Multilingual large language models (LLMs) seem to generalize somewhat across languages. We hypothesize this is a result of implicit vector space alignment. Evaluating such alignment, we see that larger models exhibit ver…

Word Embeddings