Using semi-experts to derive judgments on word sense alignment: a pilot study
The overall goal of this project is to evaluate the performance of word sense alignment (WSA) systems, focusing on obtaining examples appropriate to language learners. Building a gold standard dataset based on human expert judgments is costly in time and labor, and thus we gauge the utility of using semi-experts in performing the annotation. In an online survey, we present a sense of a target word from one dictionary with senses from the other dictionary, asking for judgments of relatedness. We note the difficulty of agreement, yet the utility in using such results to evaluate WSA work. We find that one's treatment of related senses heavily impacts the results for WSA.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationWord Sense DisambiguationSimilar Papers 제목 키워드 기반
When Polysemy Matters: Modeling Semantic Categorization with Word Embeddings
Recent work using word embeddings to model semantic categorization have indicated that static models outperform the more recent contextual class of models (Majewska et al, 2021). In this paper, we consider polysemy as a …
Word EmbeddingsThe DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change
We present the DURel tool that implements the annotation of semantic proximity between uses of words into an online, open source interface. The tool supports standardized human annotation as well as computational annotat…
ClusteringGraph ClusteringContextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge
Understanding context-dependent variation in word meanings is a key aspect of human language comprehension supported by the lexicon. Lexicographic resources (e.g., WordNet) capture only some of this context-dependent var…
Word EmbeddingsWord Sense Inventories by Non-Experts.
In this paper, we explore different strategies for implementing a crowdsourcing methodology for a single-step construction of an empirically-derived sense inventory and the corresponding sense-annotated corpus. We report…
Word Sense DisambiguationComing to its senses: Lessons learned from Approximating Retrofitted BERT representations for Word Sense information
Retrofitting static vector space word representations using external knowledge bases has yielded substantial improvements in their lexical-semantic capacities but is non-trivial to apply to contextual word embeddings (CW…
Word Embeddings