Simulating Human Gaze with Neural Visual Attention
Existing models of human visual attention are generally unable to incorporate direct task guidance and therefore cannot model an intent or goal when exploring a scene. To integrate guidance of any downstream visual task into attention modeling, we propose the Neural Visual Attention (NeVA) algorithm. To this end, we impose to neural networks the biological constraint of foveated vision and train an attention mechanism to generate visual explorations that maximize the performance with respect to the downstream task. We observe that biologically constrained neural networks generate human-like scanpaths without being trained for this objective. Extensive experiments on three common benchmark datasets show that our method outperforms state-of-the-art unsupervised human attention models in generating human-like scanpaths.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
GazeVLM: Active Vision via Internal Attention Control for Multimodal Reasoning
Human visual reasoning is governed by active vision, a process where metacognitive control drives top-down goal-directed attention, dynamically routing foveal focus toward task-relevant details while maintaining peripher…
Multimodal ReasoningSpatial ReasoningVisual ReasoningA Transformer-Based Model for the Prediction of Human Gaze Behavior on Videos
Eye-tracking applications that utilize the human gaze in video understanding tasks have become increasingly important. To effectively automate the process of video analysis based on eye-tracking data, it is important to …
Activity RecognitionGaze PredictionVideo UnderstandingSeeing with Humans: Gaze-Assisted Neural Image Captioning
Gaze reflects how humans process visual scenes and is therefore increasingly used in computer vision systems. Previous works demonstrated the potential of gaze for object-centric tasks, such as object localization and re…
Image CaptioningObjectObject LocalizationScene Recognition+1Eyes on Target: Gaze-Aware Object Detection in Egocentric Video
Human gaze offers rich supervisory signals for understanding visual attention in complex visual environments. In this paper, we propose Eyes on Target, a novel depth-aware and gaze-guided object detection framework desig…
Object DetectionAGIL: Learning Attention from Human for Visuomotor Tasks
When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze. A wealth of information r…
Atari GamesDecision MakingImitation Learning