Collateral facilitation in humans and language models
Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, with more predictable words being processed more easily. However, evidence also shows that humans display a similar processing advantage for highly anomalous words when these words are semantically related to the preceding context or to the most probable continuation. Using stimuli from 3 psycholinguistic experiments, we find that this is also almost always also the case for 8 contemporary transformer language models (BERT, ALBERT, RoBERTa, XLM-R, GPT-2, GPT-Neo, GPT-J, and XGLM). We then discuss the implications of this phenomenon for our understanding of both human language comprehension and the predictions made by language models.
Code (1)
Tasks
XLM-RMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?
Despite being designed for performance rather than cognitive plausibility, transformer language models have been found to be better at predicting metrics used to assess human language comprehension than language models w…
Sensitivities and Hedging of the Collateral Choice Option
The collateral choice option allows a collateral-posting party the opportunity to change the type of security in which the collateral is deposited. Due to non-zero collateral basis spreads, this optionality significantly…
ManagementDiscounting with Imperfect Collateral
Cash collateral is perfect in that it provides simultaneous counterparty credit risk protection and derivatives funding. Securities are imperfect collateral, because of collateral segregation or differences in CSA haircu…
Cheapest-to-Deliver Collateral: A Common Factor Approach
The collateral choice option gives the collateral posting party the opportunity to switch between different collateral currencies which is well-known to impact the asset price. Quantification of the option's value is of …
A New Model for Pricing Collateralized Financial Derivatives
This paper presents a new model for pricing financial derivatives subject to collateralization. It allows for collateral arrangements adhering to bankruptcy laws. As such, the model can back out the market price of a col…
model