Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings
Creating accurate meta-embeddings from pre-trained source embeddings has received attention lately. Methods based on global and locally-linear transformation and concatenation have shown to produce accurate meta-embeddings. In this paper, we show that the arithmetic mean of two distinct word embedding sets yields a performant meta-embedding that is comparable or better than more complex meta-embedding learning methods. The result seems counter-intuitive given that vector spaces in different source embeddings are not comparable and cannot be simply averaged. We give insight into why averaging can still produce accurate meta-embedding despite the incomparability of the source vector spaces.
Code (1)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Return of Frustratingly Easy Unsupervised Video Domain Adaptation
Unsupervised video domain adaptation (UVDA) is a practical but under-explored problem. In this paper, we propose a frustratingly easy UVDA method, called MetaTrans. Specifically, MetaTrans adopts a concise learning objec…
Action RecognitionDomain AdaptationOn the Effectiveness of Sentence Encoding for Intent Detection Meta-Learning
Recent studies on few-shot intent detection have attempted to formulate the task as a meta-learning problem, where a meta-learning model is trained with a certain capability to quickly adapt to newly specified few-shot t…
Intent DetectionMeta-LearningSemantic SimilaritySemantic Textual Similarity+4Minimizing Memorization in Meta-learning: A Causal Perspective
Meta-learning has emerged as a potent paradigm for quick learning of few-shot tasks, by leveraging the meta-knowledge learned from meta-training tasks. Well-generalized meta-knowledge that facilitates fast adaptation in …
Causal InferenceMemorizationMeta-LearningJoint Embedding of Meta-Path and Meta-Graph for Heterogeneous Information Networks
Meta-graph is currently the most powerful tool for similarity search on heterogeneous information networks,where a meta-graph is a composition of meta-paths that captures the complex structural information. However, curr…
Network EmbeddingTensor DecompositionFrustratingly Easy Uncertainty Estimation for Distribution Shift
Distribution shift is an important concern in deep image classification, produced either by corruption of the source images, or a complete change, with the solution involving domain adaptation. While the primary goal is …
Domain Adaptationimage-classificationImage ClassificationUnsupervised Domain Adaptation