Analyzing Semantic Change through Lexical Replacements
Modern language models are capable of contextualizing words based on their surrounding context. However, this capability is often compromised due to semantic change that leads to words being used in new, unexpected contexts not encountered during pre-training. In this paper, we model \textit{semantic change} by studying the effect of unexpected contexts introduced by \textit{lexical replacements}. We propose a \textit{replacement schema} where a target word is substituted with lexical replacements of varying relatedness, thus simulating different kinds of semantic change. Furthermore, we leverage the replacement schema as a basis for a novel \textit{interpretable} model for semantic change. We are also the first to evaluate the use of LLaMa for semantic change detection.
Code (1)
Tasks
Change DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Investigating Lexical Replacements for Arabic-English Code-Switched Data Augmentation
Data sparsity is a main problem hindering the development of code-switching (CS) NLP systems. In this paper, we investigate data augmentation techniques for synthesizing dialectal Arabic-English CS text. We perform lexic…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Data AugmentationLanguage Modelling+4Large-scale cloze evaluation reveals that token prediction tasks are neither lexically nor semantically aligned
In this work we compare the generative behavior at the next token prediction level in several language models by comparing them to human productions in the cloze task. We find that while large models trained for longer a…
Robustness of Misinformation Classification Systems to Adversarial Examples Through BeamAttack
We extend BeamAttack, an adversarial attack algorithm designed to evaluate the robustness of text classification systems through word-level modifications guided by beam search. Our extensions include support for word del…
Adversarial AttackMisinformationtext-classificationText ClassificationData Augmentation Techniques for Machine Translation of Code-Switched Texts: A Comparative Study
Code-switching (CSW) text generation has been receiving increasing attention as a solution to address data scarcity. In light of this growing interest, we need more comprehensive studies comparing different augmentation …
Data AugmentationMachine TranslationText GenerationTranslationMatching Meaning at Scale: Evaluating Semantic Search for 18th-Century Intellectual History through the Case of Locke
While digitized corpora have transformed the study of intellectual transmission, current methods rely heavily on lexical text reuse detection, capturing verbatim quotations but fundamentally missing paraphrases and compl…
Semantic Retrieval