An Attentive Survey of Attention Models
Attention Model has now become an important concept in neural networks that has been researched within diverse application domains. This survey provides a structured and comprehensive overview of the developments in modeling attention. In particular, we propose a taxonomy which groups existing techniques into coherent categories. We review salient neural architectures in which attention has been incorporated, and discuss applications in which modeling attention has shown a significant impact. We also describe how attention has been used to improve the interpretability of neural networks. Finally, we discuss some future research directions in attention. We hope this survey will provide a succinct introduction to attention models and guide practitioners while developing approaches for their applications.
Code (0)
등록된 구현이 없습니다.
Tasks
SurveyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning to Pay Attention: Unsupervised Modeling of Attentive and Inattentive Respondents in Survey Data
The integrity of behavioral and social-science surveys depends on detecting inattentive respondents who provide random or low-effort answers. Traditional safeguards, such as attention checks, are often costly, reactive, …
Neural Attention for Image Captioning: Review of Outstanding Methods
Image captioning is the task of automatically generating sentences that describe an input image in the best way possible. The most successful techniques for automatically generating image captions have recently used atte…
DecoderDeep LearningImage CaptioningModeling User Behavior from Adaptive Surveys with Supplemental Context
Modeling user behavior is critical across many industries where understanding preferences, intent, or decisions informs personalization, targeting, and strategic outcomes. Surveys have long served as a classical mechanis…
Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
In NLP, convolutional neural networks (CNNs) have benefited less than recurrent neural networks (RNNs) from attention mechanisms. We hypothesize that this is because the attention in CNNs has been mainly implemented as a…
Claim VerificationNatural Language InferenceRepresentation LearningSentence+1Modelling Sentence Pairs with Tree-structured Attentive Encoder
We describe an attentive encoder that combines tree-structured recursive neural networks and sequential recurrent neural networks for modelling sentence pairs. Since existing attentive models exert attention on the seque…
Paraphrase IdentificationQuestion SelectionSemantic SimilaritySemantic Textual Similarity+1