Capturing Evolution in Word Usage: Just Add More Clusters?
The way the words are used evolves through time, mirroring cultural or technological evolution of society. Semantic change detection is the task of detecting and analysing word evolution in textual data, even in short periods of time. In this paper we focus on a new set of methods relying on contextualised embeddings, a type of semantic modelling that revolutionised the NLP field recently. We leverage the ability of the transformer-based BERT model to generate contextualised embeddings capable of detecting semantic change of words across time. Several approaches are compared in a common setting in order to establish strengths and weaknesses for each of them. We also propose several ideas for improvements, managing to drastically improve the performance of existing approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Change DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport
Lexical semantic change detection aims to identify shifts in word meanings over time. While existing methods using embeddings from a diachronic corpus pair estimate the degree of change for target words, they offer limit…
Change DetectionWord EmbeddingsModeling Sense Structure in Word Usage Graphs with the Weighted Stochastic Block Model
We suggest to model human-annotated Word Usage Graphs capturing fine-grained semantic proximity distinctions between word uses with a Bayesian formulation of the Weighted Stochastic Block Model, a generative model for ra…
Stochastic Block ModelImproving Interpretability of Word Embeddings by Generating Definition and Usage
Word embeddings are substantially successful in capturing semantic relations among words. However, these lexical semantics are difficult to be interpreted. Definition modeling provides a more intuitive way to evaluate em…
Multi-Task LearningWord EmbeddingsJust In Time Transformers
Precise energy load forecasting in residential households is crucial for mitigating carbon emissions and enhancing energy efficiency; indeed, accurate forecasting enables utility companies and policymakers, who advocate …
energy managementLoad ForecastingManagementMultiple evolutionary pressures shape identical consonant avoidance in the world's languages
Languages disfavor word forms containing sequences of similar or identical consonants, due to the biomechanical and cognitive difficulties posed by patterns of this sort. However, the specific evolutionary processes resp…