Do pretrained transformers infer telicity like humans?
Pretrained transformer-based language models achieve state-of-the-art performance in many NLP tasks, but it is an open question whether the knowledge acquired by the models during pretraining resembles the linguistic knowledge of humans. We present both humans and pretrained transformers with descriptions of events, and measure their preference for telic interpretations (the event has a natural endpoint) or atelic interpretations (the event does not have a natural endpoint). To measure these preferences and determine what factors influence them, we design an English test and a novel-word test that include a variety of linguistic cues (noun phrase quantity, resultative structure, contextual information, temporal units) that bias toward certain interpretations. We find that humans’ choice of telicity interpretation is reliably influenced by theoretically-motivated cues, transformer models (BERT and RoBERTa) are influenced by some (though not all) of the cues, and transformer models often rely more heavily on temporal units than humans do.
Code (0)
등록된 구현이 없습니다.
Tasks
Open-Ended Question AnsweringSimilar Papers 제목 키워드 기반
About Time: Do Transformers Learn Temporal Verbal Aspect?
Aspect is a linguistic concept that describes how an action, event, or state of a verb phrase is situated in time. In this paper, we explore whether different transformer models are capable of identifying aspectual featu…
About Time: Do Transformers Learn Temporal Verbal Aspect?
Aspect is a linguistic concept that describes how an action, event, or state of a verb phrase is situated in time. In this paper, we explore whether different transformer models are capable of identifying aspectual featu…
Interpretable dimensions support an effect of agentivity and telicity on split intransitivity
Intransitive verbs fall into two different syntactic classes, unergatives and unaccusatives. It has long been argued that verbs describing an agentive action are more likely to appear in an unergative syntax, and those d…
Classification of telicity using cross-linguistic annotation projection
This paper addresses the automatic recognition of telicity, an aspectual notion. A telic event includes a natural endpoint ({``}she walked home{''}), while an atelic event does not ({``}she walked around{''}). Recognizin…
ClassificationGeneral ClassificationMachine TranslationNatural Language Understanding+2Emergent Ordinal Geometry in Transformers Trained on Local Comparisons
Transitive inference is the challenge of inferring that A < C from knowing only adjacent relations (A < B, B < C). It is solved by humans and animals not through logical chaining but via an analogue mental number line, w…