Word class flexibility: A deep contextualized approach
Word class flexibility refers to the phenomenon whereby a single word form is used across different grammatical categories. Extensive work in linguistic typology has sought to characterize word class flexibility across languages, but quantifying this phenomenon accurately and at scale has been fraught with difficulties. We propose a principled methodology to explore regularity in word class flexibility. Our method builds on recent work in contextualized word embeddings to quantify semantic shift between word classes (e.g., noun-to-verb, verb-to-noun), and we apply this method to 37 languages. We find that contextualized embeddings not only capture human judgment of class variation within words in English, but also uncover shared tendencies in class flexibility across languages. Specifically, we find greater semantic variation when flexible lemmas are used in their dominant word class, supporting the view that word class flexibility is a directional process. Our work highlights the utility of deep contextualized models in linguistic typology.
Code (2)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Contextualized Weak Supervision for Text Classification
Weakly supervised text classification based on a few user-provided seed words has recently attracted much attention from researchers. Existing methods mainly generate pseudo-labels in a context-free manner (e.g., string …
ClassificationGeneral Classificationtext-classificationText ClassificationRetrofitting Contextualized Word Embeddings with Paraphrases
Contextualized word embedding models, such as ELMo, generate meaningful representations of words and their context. These models have been shown to have a great impact on downstream applications. However, in many cases, …
SentenceSentence ClassificationWord EmbeddingsLeveraging Three Types of Embeddings from Masked Language Models in Idiom Token Classification
Many linguistic expressions have idiomatic and literal interpretations, and the automatic distinction of these two interpretations has been studied for decades. Recent research has shown that contextualized word embeddin…
Classificationtoken-classificationToken ClassificationWord EmbeddingsThe Role of Protected Class Word Lists in Bias Identification of Contextualized Word Representations
Systemic bias in word embeddings has been widely reported and studied, and efforts made to debias them; however, new contextualized embeddings such as ELMo and BERT are only now being similarly studied. Standard debiasin…
Word EmbeddingsRevisiting Word Embeddings in the LLM Era
Large Language Models (LLMs) have recently shown remarkable advancement in various NLP tasks. As such, a popular trend has emerged lately where NLP researchers extract word/sentence/document embeddings from these large d…
DecoderSentenceWord Embeddings