paper-with-me

홈 › Papers

CL-IMS @ DIACR-Ita: Volente o Nolente: BERT does not outperform SGNS on Semantic Change Detection

2020-11-14 · Severin Laicher, Gioia Baldissin, Enrique Castañeda, Dominik Schlechtweg, Sabine Schulte im Walde

We present the results of our participation in the DIACR-Ita shared task on lexical semantic change detection for Italian. We exploit Average Pairwise Distance of token-based BERT embeddings between time points and rank 5 (of 8) in the official ranking with an accuracy of $.72$. While we tune parameters on the English data set of SemEval-2020 Task 1 and reach high performance, this does not translate to the Italian DIACR-Ita data set. Our results show that we do not manage to find robust ways to exploit BERT embeddings in lexical semantic change detection.

📄 PDF Abstract BibTeX arXiv:2011.07247

Code (1)

Garrafao/TokenChange 공식 구현

Tasks

Change Detection

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Multi-Head Attention 설명 없음
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Residual Connection 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
WordPiece 설명 없음

Similar Papers 제목 키워드 기반

Diacritics Restoration using BERT with Analysis on Czech language

2021-05-24 · Jakub Náplava, Milan Straka, Jana Straková

We propose a new architecture for diacritics restoration based on contextualized embeddings, namely BERT, and we evaluate it on 12 languages with diacritics. Furthermore, we conduct a detailed error analysis on Czech, a …

Croatian Text DiacritizationCzech Text DiacritizationFrench Text DiacritizationHungarian Text Diacritization+8

MenakBERT -- Hebrew Diacriticizer

2024-10-03 · Ido Cohen, Jacob Gidron, Idan Pinto

Diacritical marks in the Hebrew language give words their vocalized form. The task of adding diacritical marks to plain Hebrew text is still dominated by a system that relies heavily on human-curated resources. Recent mo…

Part-Of-Speech Tagging

D-Nikud: Enhancing Hebrew Diacritization with LSTM and Pretrained Models

2024-01-30 · Adi Rosenthal, Nadav Shaked

D-Nikud, a novel approach to Hebrew diacritization that integrates the strengths of LSTM networks and BERT-based (transformer) pre-trained model. Inspired by the methodologies employed in Nakdimon, we integrate it with t…

The Degree of Language Diacriticity and Its Effect on Tasks

2026-03-29 · Adi Cohen, Yuval Pinter arxiv

Diacritics are orthographic marks that clarify pronunciation, distinguish similar words, or alter meaning. They play a central role in many writing systems, yet their impact on language technology has not been systematic…

Exploiting Arabic Diacritization for High Quality Automatic Annotation

2016-05-01 · LREC 2016 5 · Nizar Habash, Anas Shahrour, Muhamed Al-Khalil

We present a novel technique for Arabic morphological annotation. The technique utilizes diacritization to produce morphological annotations of quality comparable to human annotators. Although Arabic text is generally wr…

LEMMAVocal Bursts Intensity Prediction