Comparing Contextual and Static Word Embeddings with Small Data
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
Replacing static word embeddings with contextualized word representations has yielded significant improvements on many NLP tasks. However, just how contextual are the contextualized representations produced by models suc…
Word EmbeddingsWhen Polysemy Matters: Modeling Semantic Categorization with Word Embeddings
Recent work using word embeddings to model semantic categorization have indicated that static models outperform the more recent contextual class of models (Majewska et al, 2021). In this paper, we consider polysemy as a …
Word EmbeddingsLearning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy
Contextualised word embeddings generated from Neural Language Models (NLMs), such as BERT, represent a word with a vector that considers the semantics of the target word as well its context. On the other hand, static wor…
Word EmbeddingsWord Sense DisambiguationContextualized Embeddings for Enriching Linguistic Analyses on Politeness
Linguistic analyses in natural language processing (NLP) have often been performed around the static notion of words where the context (surrounding words) is not considered. For example, previous analyses on politeness h…
ClusteringWord EmbeddingsCombining static and contextualised multilingual embeddings
Static and contextual multilingual embeddings have complementary strengths. Static embeddings, while less expressive than contextual language models, can be more straightforwardly aligned across multiple languages. Conte…
RetrievalXLM-R