Closed Form Word Embedding Alignment

2018-06-04 · Sunipa Dev, Safia Hassan, Jeff M. Phillips

We develop a family of techniques to align word embeddings which are derived from different source datasets or created using different mechanisms (e.g., GloVe or word2vec). Our methods are simple and have a closed form to optimally rotate, translate, and scale to minimize root mean squared errors or maximize the average cosine similarity between two embeddings of the same vocabulary into the same dimensional space. Our methods extend approaches known as Absolute Orientation, which are popular for aligning objects in three-dimensions, and generalize an approach by Smith etal (ICLR 2017). We prove new results for optimal scaling and for maximizing cosine similarity. Then we demonstrate how to evaluate the similarity of embeddings from different sources or mechanisms, and that certain properties like synonyms and analogies are preserved across the embeddings and can be enhanced by simply aligning and averaging ensembles of embeddings.

📄 PDF Abstract BibTeX arXiv:1806.01330

Code (0)

등록된 구현이 없습니다.

Tasks

FormWord Embeddings

Methods 이 논문이 사용한 방법론

GloVe GloVe Embeddings are a type of word embedding that encode the co-occurrence probability ratio between two words as vector differences. GloVe uses a weighted least squares…

ERMI at PARSEME Shared Task 2020: Embedding-Rich Multiword Expression Identification

2020-12-01 · COLING (MWE) 2020 12 · Zeynep Yirmibeşoğlu, Tunga Güngör

This paper describes the ERMI system submitted to the closed track of the PARSEME shared task 2020 on automatic identification of verbal multiword expressions (VMWEs). ERMI is an embedding-rich bidirectional LSTM-CRF mod…

POS TAG

Learning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy

2020-06-20 · Antonio H. O. Fonseca, David van Dijk

Word translation is an integral part of language translation. In machine translation, each language is considered a domain with its own word embedding. The alignment between word embeddings allows linking semantically eq…

Machine Translation Sentence Translation Word Alignment+2

SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings

2020-04-18 · Findings of the Association for Computational Linguistics 2020 · Masoud Jalili Sabet, Philipp Dufter, François Yvon, Hinrich Schütze

Word alignments are useful for tasks like statistical and neural machine translation (NMT) and cross-lingual annotation projection. Statistical word aligners perform well, as do methods that extract alignments jointly wi…

Machine Translation Multilingual Word Embeddings NMT Translation+2

Word-Alignment-Based Segment-Level Machine Translation Evaluation using Word Embeddings

2017-04-02 · Junki Matsuo, Mamoru Komachi, Katsuhito Sudoh

One of the most important problems in machine translation (MT) evaluation is to evaluate the similarity between translation hypotheses with different surface forms from the reference, especially at the segment level. We …

Machine Translation Translation Word Alignment Word Embeddings

English-Malay Cross-Lingual Embedding Alignment using Bilingual Lexicon Augmentation

2022-05-01 · ACL 2022 5 · Ying Hao Lim, Jasy Suet Yan Liew

As high-quality Malay language resources are still a scarcity, cross lingual word embeddings make it possible for richer English resources to be leveraged for downstream Malay text classification tasks. This paper focuse…

Cross-Lingual Word Embeddings Machine Translation text-classification Text Classification+1