Homonym Identification using BERT -- Using a Clustering Approach

2021-01-07 · Rohan Saha

Homonym identification is important for WSD that require coarse-grained partitions of senses. The goal of this project is to determine whether contextual information is sufficient for identifying a homonymous word. To capture the context, BERT embeddings are used as opposed to Word2Vec, which conflates senses into one vector. SemCor is leveraged to retrieve the embeddings. Various clustering algorithms are applied to the embeddings. Finally, the embeddings are visualized in a lower-dimensional space to understand the feasibility of the clustering process.

📄 PDF Abstract BibTeX arXiv:2101.02398

Code (0)

등록된 구현이 없습니다.

Tasks

Clustering

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Weight Decay 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Attention 설명 없음

Multi-Head Attention 설명 없음

Homonym normalisation by word sense clustering: a case in Japanese

2020-12-01 · COLING 2020 8 · Yo Sato, Kevin Heffernan

This work presents a method of word sense clustering that differentiates homonyms and merge homophones, taking Japanese as an example, where orthographical variation causes problem for language processing. It uses contex…

Clustering Language Modeling Language Modelling Transliteration

Patterns of Polysemy and Homonymy in Contextualised Language Models

2021-11-01 · Findings (EMNLP) 2021 11 · Janosch Haber, Massimo Poesio

One of the central aspects of contextualised language models is that they should be able to distinguish the meaning of lexically ambiguous words by their contexts. In this paper we investigate the extent to which the con…

Patterns of Lexical Ambiguity in Contextualised Language Models

2021-09-27 · Janosch Haber, Massimo Poesio

Homonymy Information for English WordNet

2022-12-16 · gwll (LREC) 2022 6 · Rowan Hall Maudslay, Simone Teufel

A widely acknowledged shortcoming of WordNet is that it lacks a distinction between word meanings which are systematically related (polysemy), and those which are coincidental (homonymy). Several previous works have atte…

Language Modelling

Assessing Polyseme Sense Similarity through Co-predication Acceptability and Contextualised Embedding Distance

2020-12-01 · Joint Conference on Lexical and Computational Semantics 2020 · Janosch Haber, Massimo Poesio

Co-predication is one of the most frequently used linguistic tests to tell apart shifts in polysemic sense from changes in homonymic meaning. It is increasingly coming under criticism as evidence is accumulating that it …

Word Embeddings