paper-with-me

홈 › Papers

Random Attention Span

2024-05-19 · Dazhuo Wei

In this paper, I introduce a random attention span model (RAS) which uses stopping time to identify decision-makers' behavior under limited attention. Unlike many limited attention models, the RAS identifies preferences using time variation without any need for menu variation. In addition, the RAS allows the consideration set to be correlated with the preference. I also use the revealed preference theory that provides testable implications for observable choice probabilities. Then, I test the model and estimate the preference distribution using data from M-Turk experiments on choice behaviors that involve lotteries; there is general alignment with the distribution results from logit attention model.

📄 PDF Abstract BibTeX arXiv:2405.11578

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

Revenue Maximization and Learning in Products Ranking

2020-12-07 · Ningyuan Chen, Anran Li, Shuoguang Yang

We consider the revenue maximization problem for an online retailer who plans to display in order a set of products differing in their prices and qualities. Consumers have attention spans, i.e., the maximum number of pro…

Superlinear Multi-Step Attention

2026-01-26 · Yufeng Huang arxiv

In this paper, we propose \textbf{Superlinear attention}, a fully trainable multi-step attention architecture that achieves subquadratic complexity for long sequences while preserving \textbf{random context access} (a.k.…

The Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection

2026-06-14 · Emma Leonhart arxiv

A feasibility and dynamics study of the Reservoir Attention Network (RAN), an architecture that injects a fixed, randomly-initialized reservoir into the mid-layer attention of a pretrained transformer to carry state acro…

FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention

2026-07-08 · Athanasios Zeris arxiv

FFT-based spectral preprocessing of learned query-key (Q/K) projections substantially improves transformer attention on character-level language modelling. On TinyShakespeare: a fixed random spectral filter achieves val=…

Language Modelling

SpanBERT: Improving Pre-training by Representing and Predicting Spans

2019-07-24 · TACL 2020 1 · Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld 외

We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the…

Coreference ResolutionLinguistic AcceptabilityNatural Language InferenceOpen-Domain Question Answering+6