Learning Meta-Embeddings by Using Ensembles of Embedding Sets
Word embeddings -- distributed representations of words -- in deep learning are beneficial for many tasks in natural language processing (NLP). However, different embedding sets vary greatly in quality and characteristics of the captured semantics. Instead of relying on a more advanced algorithm for embedding learning, this paper proposes an ensemble approach of combining different public embedding sets with the aim of learning meta-embeddings. Experiments on word similarity and analogy tasks and on part-of-speech tagging show better performance of meta-embeddings compared to individual embedding sets. One advantage of meta-embeddings is the increased vocabulary coverage. We will release our meta-embeddings publicly.
Code (1)
Tasks
Part-Of-Speech TaggingWord EmbeddingsWord SimilaritySimilar Papers 제목 키워드 기반
Dynamic Meta-Embeddings for Improved Sentence Representations
While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce d…
SentenceWord EmbeddingsFrustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings
Creating accurate meta-embeddings from pre-trained source embeddings has received attention lately. Methods based on global and locally-linear transformation and concatenation have shown to produce accurate meta-embeddin…
Word EmbeddingsLearning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings
Given multiple source word embeddings learnt using diverse algorithms and lexical resources, meta word embedding learning methods attempt to learn more accurate and wide-coverage word embeddings. Prior work on meta-embed…
Word EmbeddingsBenchmarking Meta-embeddings: What Works and What Does Not
In the last few years, several methods have been proposed to build meta-embeddings. The general aim was to obtain new representations integrating complementary knowledge from different source pre-trained embeddings there…
BenchmarkingEmbeddings EvaluationTogether We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddings
Sense embedding learning methods learn multiple vectors for a given ambiguous word, corresponding to its different word senses. For this purpose, different methods have been proposed in prior work on sense embedding lear…
Word Sense Disambiguation