black[LSCDiscovery shared task] GlossReader at LSCDiscovery: Train to Select a Proper Gloss in English – Discover Lexical Semantic Change in Spanish
The contextualized embeddings obtained from neural networks pre-trained as Language Models (LM) or Masked Language Models (MLM) are not well suitable for solving the Lexical Semantic Change Detection (LSCD) task because they are more sensitive to changes in word forms rather than word meaning, a property previously known as the word form bias or orthographic bias. Unlike many other NLP tasks, it is also not obvious how to fine-tune such models for LSCD. In order to conclude if there are any differences between senses of a particular word in two corpora, a human annotator or a system shall analyze many examples containing this word from both corpora. This makes annotation of LSCD datasets very labour-consuming. The existing LSCD datasets contain up to 100 words that are labeled according to their semantic change, which is hardly enough for fine-tuning. To solve these problems we fine-tune the XLM-R MLM as part of a gloss-based WSD system on a large WSD dataset in English. Then we employ zero-shot cross-lingual transferability of XLM-R to build the contextualized embeddings for examples in Spanish. In order to obtain the graded change score for each word, we calculate the average distance between our improved contextualized embeddings of its old and new occurrences. For the binary change detection subtask, we apply thresholding to the same scores. Our solution has shown the best results among all other participants in all subtasks except for the optional sense gain detection subtask.
Code (0)
등록된 구현이 없습니다.
Tasks
Change DetectionXLM-RMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
black[LSCDiscovery shared task] BOS at LSCDiscovery: Lexical Substitution for Interpretable Lexical Semantic Change Detection
We propose a solution for the LSCDiscovery shared task on Lexical Semantic Change Detection in Spanish. Our approach is based on generating lexical substitutes that describe old and new senses of a given word. This appro…
Change Detectionblack[LSCDiscovery shared task] CoToHiLi at LSCDiscovery: the Role of Linguistic Features in Predicting Semantic Change
This paper presents the contributions of the CoToHiLi team for the LSCDiscovery shared task on semantic change in the Spanish language. We participated in both tasks (graded discovery and binary change, including sense g…
black[LSCDiscovery shared task] DeepMistake at LSCDiscovery: Can a Multilingual Word-in-Context Model Replace Human Annotators?
In this paper we describe our solution of the LSCDiscovery shared task on Lexical Semantic Change Discovery (LSCD) in Spanish. Our solution employs a Word-in-Context (WiC) model, which is trained to determine if a partic…
Change Detectionblack[LSCDiscovery shared task] LSCDiscovery: A shared task on semantic change discovery and detection in Spanish
We present the first shared task on semantic change discovery and detection in Spanish. We create the first dataset of Spanish words manually annotated by semantic change using the DURel framewok (Schlechtweg et al., 201…
Change Detectionblack[LSCDiscovery shared task] UAlberta at LSCDiscovery: Lexical Semantic Change Detection via Word Sense Disambiguation
We describe our two systems for the shared task on Lexical Semantic Change Discovery in Spanish. For binary change detection, we frame the task as a word sense disambiguation (WSD) problem. We derive sense frequency dist…
Change DetectionWord Sense Disambiguation