Contextualized Word Vector-based Methods for Discovering Semantic Differences with No Training nor Word Alignment
In this paper, we propose methods for discovering semantic differences in words appearing in two corpora based on the norms of contextualized word vectors. The key idea is that the coverage of meanings is reflected in the norm of its mean word vector. The proposed methods do not require the assumptions concerning words and corpora for comparison that the previous methods do. All they require are to compute the mean vector of contextualized word vectors and its norm for each word type. Nevertheless, they are (i) robust for the skew in corpus size; (ii) capable of detecting semantic differences in infrequent words; and (iii) effective in pinpointing word instances that have a meaning missing in one of the two corpora for comparison. We show these advantages for native and non-native English corpora and also for historical corpora.
Code (0)
등록된 구현이 없습니다.
Tasks
Word AlignmentSimilar Papers 제목 키워드 기반
Augmenting Chinese WordNet semantic relations with contextualized embeddings
Constructing semantic relations in WordNet has been a labour-intensive task, especially in a dynamic and fast-changing language environment. Combined with recent advancements of contextualized embeddings, this paper prop…
Discovering Differences in the Representation of People using Contextualized Semantic Axes
A common paradigm for identifying semantic differences across social and temporal contexts is the use of static word embeddings and their distances. In particular, past work has compared embeddings against "semantic axes…
Word EmbeddingsVec2Gloss: definition modeling leveraging contextualized vectors with Wordnet gloss
Contextualized embeddings are proven to be powerful tools in multiple NLP tasks. Nonetheless, challenges regarding their interpretability and capability to represent lexical semantics still remain. In this paper, we prop…
Unsupervised Distillation of Syntactic Information from Contextualized Word Representations
Contextualized word representations, such as ELMo and BERT, were shown to perform well on various semantic and syntactic tasks. In this work, we tackle the task of unsupervised disentanglement between semantics and struc…
DisentanglementMetric LearningDeriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection
One of the long-standing challenges in lexical semantics consists in learning representations of words which reflect their semantic properties. The remarkable success of word embeddings for this purpose suggests that hig…
SentenceWord Embeddings