On the Curious Case of $\ell_2$ norm of Sense Embeddings
We show that the $\ell_2$ norm of a static sense embedding encodes information related to the frequency of that sense in the training corpus used to learn the sense embeddings. This finding can be seen as an extension of a previously known relationship for word embeddings to sense embeddings. Our experimental results show that, in spite of its simplicity, the $\ell_2$ norm of sense embeddings is a surprisingly effective feature for several word sense related tasks such as (a) most frequent sense prediction, (b) Word-in-Context (WiC), and (c) Word Sense Disambiguation (WSD). In particular, by simply including the $\ell_2$ norm of a sense embedding as a feature in a classifier, we show that we can improve WiC and WSD methods that use static sense embeddings.
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsWord Sense DisambiguationSimilar Papers 제목 키워드 기반
Word Embeddings vs Word Types for Sequence Labeling: the Curious Case of CV Parsing
Homonym normalisation by word sense clustering: a case in Japanese
This work presents a method of word sense clustering that differentiates homonyms and merge homophones, taking Japanese as an example, where orthographical variation causes problem for language processing. It uses contex…
ClusteringLanguage ModelingLanguage ModellingTransliterationCurious Explorer: a provable exploration strategy in Policy Learning
Having access to an exploring restart distribution (the so-called wide coverage assumption) is critical with policy gradient methods. This is due to the fact that, while the objective function is insensitive to updates i…
Policy Gradient MethodsCommon Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs
Large Language Models (LLMs) are increasingly deployed across diverse real-world applications and user communities. As such, it is crucial that these models remain both morally grounded and knowledge-aware. In this work,…
Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word. One sense of an ambiguous word might be socially biased while its other senses remain unbiased. In comparison to …
Word Embeddings