Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension
While neural networks with attention mechanisms have achieved superior performance on many natural language processing tasks, it remains unclear to which extent learned attention resembles human visual attention. We study the similarity between human visual and neural attention and analyze if neural attention-based methods perform better if they mimic human attention. To this end, we compare state-of-the-art networks based on long short-term memory (LSTM), convolutional neural (CNN) and XLNet Transformer architectures on a question answering task. We evaluate all methods on a novel 23-participant dataset of eye tracking data recorded while reading movie plots. We find that while higher similarity to human attention and performance significantly correlates to the LSTM and CNN this does not hold true for the XLNet -- despite the fact that the XLNet performs best on this challenging task. Our work not only shows that different architectures seem to learn rather different neural attention but also that similarity of neural to human attention is not necessarily helpful and hence desirable.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionQuestion AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension
While neural networks with attention mechanisms have achieved superior performance on many natural language processing tasks, it remains unclear to which extent learned attention resembles human visual attention. In this…
Machine Reading ComprehensionReading ComprehensionUnderstanding More about Human and Machine Attention in Deep Neural Networks
Human visual system can selectively attend to parts of a scene for quick perception, a biological mechanism known as Human attention. Inspired by this, recent deep learning models encode attention mechanisms to focus on …
Fine-Grained Image ClassificationImage ClassificationSemantic SegmentationTemporal Action LocalizationTeaching Machines to Code: Neural Markup Generation with Visual Attention
We present a neural transducer model with visual attention that learns to generate LaTeX markup of a real-world math formula given its image. Applying sequence modeling and transduction techniques that have been very suc…
MathOptical Character Recognition (OCR)From Black Box to Bijection: Interpreting Machine Learning to Build a Zeta Map Algorithm
There is a large class of problems in algebraic combinatorics which can be distilled into the same challenge: construct an explicit combinatorial bijection. Traditionally, researchers have solved challenges like these by…
Interpretation of Feature Space using Multi-Channel Attentional Sub-Networks
Convolutional Neural Networks have achieved impressive results in various tasks, but interpreting the internal mechanism is a challenging problem. To tackle this problem, we exploit a multi-channel attention mechanism in…
Attribute