Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection
Semantic change detection concerns the task of identifying words whose meaning has changed over time. Current state-of-the-art approaches operating on neural embeddings detect the level of semantic change in a word by comparing its vector representation in two distinct time periods, without considering its evolution through time. In this work, we propose three variants of sequential models for detecting semantically shifted words, effectively accounting for the changes in the word representations over time. Through extensive experimentation under various settings with synthetic and real data we showcase the importance of sequential modelling of word vectors through time for semantic change detection. Finally, we compare different approaches in a quantitative manner, demonstrating that temporal modelling of word representations yields a clear-cut advantage in performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Change DetectionSimilar Papers 제목 키워드 기반
Autoencoding Word Representations through Time for Semantic Change Detection
Semantic change detection concerns the task of identifying words whose meaning has changed over time. The current state-of-the-art detects the level of semantic change in a word by comparing its vector representation in …
Change DetectionCapturing Evolution in Word Usage: Just Add More Clusters?
The way the words are used evolves through time, mirroring cultural or technological evolution of society. Semantic change detection is the task of detecting and analysing word evolution in textual data, even in short pe…
Change DetectionSequential Modelling with Applications to Music Recommendation, Fact-Checking, and Speed Reading
Sequential modelling entails making sense of sequential data, which naturally occurs in a wide array of domains. One example is systems that interact with users, log user actions and behaviour, and make recommendations o…
Fact CheckingMusic RecommendationSentenceCascaded Semantic and Positional Self-Attention Network for Document Classification
Transformers have shown great success in learning representations for language modelling. However, an open challenge still remains on how to systematically aggregate semantic information (word embedding) with positional …
ClassificationDocument ClassificationGeneral ClassificationLanguage ModellingLanguage Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation
Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work…
Language ModellingLEMMAWord EmbeddingsWord Sense Disambiguation