Stress Test for BERT and Deep Models: Predicting Words from Italian Poetry
In this paper we present a set of experiments carried out with BERT on a number of Italian sentences taken from poetry domain. The experiments are organized on the hypothesis of a very high level of difficulty in predictability at the three levels of linguistic complexity that we intend to monitor: lexical, syntactic and semantic level. To test this hypothesis we ran the Italian version of BERT with 80 sentences for a total of 900 tokens mostly extracted from Italian poetry of the first half of last century. Then we alternated canonical and noncanonical versions of the same sentence before processing them with the same DL model. We used then sentences from the newswire domain containing similar syntactic structures. The results show that the DL model is highly sensitive to presence of noncanonical structures. However, DLs are also very sensitive to word frequency and to local non literal meaning compositional effect. This is also apparent by the preference for predicting function vs content words, collocates vs infrequent word phrases. In the paper, we focused our attention on the use of subword units done by BERT for out of vocabulary words.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Emotions in Macroeconomic News and their Impact on the European Bond Market
We show how emotions extracted from macroeconomic news can be used to explain and forecast future behaviour of sovereign bond yield spreads in Italy and Spain. We use a big, open-source, database known as Global Database…
GiusBERTo: A Legal Language Model for Personal Data De-identification in Italian Court of Auditors Decisions
Recent advances in Natural Language Processing have demonstrated the effectiveness of pretrained language models like BERT for a variety of downstream tasks. We present GiusBERTo, the first BERT-based model specialized f…
De-identificationLanguage ModelingLanguage ModellingPredicting municipalities in financial distress: a machine learning approach enhanced by domain expertise
Financial distress of municipalities, although comparable to bankruptcy of private companies, has a far more serious impact on the well-being of communities. For this reason, it is essential to detect deficits as soon as…
StressRoBERTa: Cross-Condition Transfer Learning from Depression, Anxiety, and PTSD to Stress Detection
The prevalence of chronic stress represents a significant public health concern, with social media platforms like Twitter serving as important venues for individuals to share their experiences. This paper introduces Stre…
Transfer LearningUnsupervised Law Article Mining based on Deep Pre-Trained Language Representation Models with Application to the Italian Civil Code
Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa…
ArticlesAttributeFew-Shot LearningPrediction+1