Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension
While neural networks with attention mechanisms have achieved superior performance on many natural language processing tasks, it remains unclear to which extent learned attention resembles human visual attention. In this paper, we propose a new method that leverages eye-tracking data to investigate the relationship between human visual attention and neural attention in machine reading comprehension. To this end, we introduce a novel 23 participant eye tracking dataset - MQA-RC, in which participants read movie plots and answered pre-defined questions. We compare state of the art networks based on long short-term memory (LSTM), convolutional neural models (CNN) and XLNet Transformer architectures. We find that higher similarity to human attention and performance significantly correlates to the LSTM and CNN models. However, we show this relationship does not hold true for the XLNet models -- despite the fact that the XLNet performs best on this challenging task. Our results suggest that different architectures seem to learn rather different neural attention strategies and similarity of neural to human attention does not guarantee best performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionReading ComprehensionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension
While neural networks with attention mechanisms have achieved superior performance on many natural language processing tasks, it remains unclear to which extent learned attention resembles human visual attention. We stud…
Machine Reading ComprehensionQuestion AnsweringReading ComprehensionUnderstanding More about Human and Machine Attention in Deep Neural Networks
Human visual system can selectively attend to parts of a scene for quick perception, a biological mechanism known as Human attention. Inspired by this, recent deep learning models encode attention mechanisms to focus on …
Fine-Grained Image ClassificationImage ClassificationSemantic SegmentationTemporal Action LocalizationTeaching Machines to Code: Neural Markup Generation with Visual Attention
We present a neural transducer model with visual attention that learns to generate LaTeX markup of a real-world math formula given its image. Applying sequence modeling and transduction techniques that have been very suc…
MathOptical Character Recognition (OCR)From Black Box to Bijection: Interpreting Machine Learning to Build a Zeta Map Algorithm
There is a large class of problems in algebraic combinatorics which can be distilled into the same challenge: construct an explicit combinatorial bijection. Traditionally, researchers have solved challenges like these by…
Interpretation of Feature Space using Multi-Channel Attentional Sub-Networks
Convolutional Neural Networks have achieved impressive results in various tasks, but interpreting the internal mechanism is a challenging problem. To tackle this problem, we exploit a multi-channel attention mechanism in…
Attribute