Generalized Measures of Anticipation and Responsivity in Online Language Processing
We introduce a generalization of classic information-theoretic measures of predictive uncertainty in online language processing, based on the simulation of expected continuations of incremental linguistic contexts. Our framework provides a formal definition of anticipatory and responsive measures, and it equips experimenters with the tools to define new, more expressive measures beyond standard next-symbol entropy and surprisal. While extracting these standard quantities from language models is convenient, we demonstrate that using Monte Carlo simulation to estimate alternative responsive and anticipatory measures pays off empirically: New special cases of our generalized formula exhibit enhanced predictive power compared to surprisal for human cloze completion probability as well as ELAN, LAN, and N400 amplitudes, and greater complementarity with surprisal in predicting reading times.
Code (1)
Similar Papers 제목 키워드 기반
Computational Analysis of Conversation Dynamics through Participant Responsivity
Growing literature explores toxicity and polarization in discourse, with comparatively less work on characterizing what makes dialogue prosocial and constructive. We explore conversational discourse and investigate a met…
Semantic SimilarityMemory-and-Anticipation Transformer for Online Action Understanding
Most existing forecasting systems are memory-based methods, which attempt to mimic human forecasting ability by employing various memory mechanisms and have progressed in temporal modeling for memory dependency. Neverthe…
Action DetectionAction UnderstandingOnline Action DetectionParaFIS:A new online fuzzy inference system based on parallel drift anticipation
This paper proposes a new architecture of incremen-tal fuzzy inference system (also called Evolving Fuzzy System-EFS). In the context of classifying data stream in non stationary environment, concept drifts problems must…
From Detection to Anticipation: Online Understanding of Struggles across Various Tasks and Activities
Understanding human skill performance is essential for intelligent assistive systems, with struggle recognition offering a natural cue for identifying user difficulties. While prior work focuses on offline struggle class…
Anticipating human actions by correlating past with the future with Jaccard similarity measures
We propose a framework for early action recognition and anticipation by correlating past features with the future using three novel similarity measures called Jaccard vector similarity, Jaccard cross-correlation and Jacc…
Action AnticipationAction Recognition