What can computational models learn from human selective attention? A review from an audiovisual crossmodal perspective
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for computational intelligent agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning to See What You Need: Gaze Attention for Multimodal Large Language Models
When humans describe a visual scene, they do not process the entire image uniformly; instead, they selectively fixate on regions relevant to their intended description. In contrast, current multimodal large language mode…
WW-Nets: Dual Neural Networks for Object Detection
We propose a new deep convolutional neural network framework that uses object location knowledge implicit in network connection weights to guide selective attention in object detection tasks. Our approach is called What-…
Objectobject-detectionObject DetectionBoosted Attention: Leveraging Human Attention for Image Captioning
Visual attention has shown usefulness in image captioning, with the goal of enabling a caption model to selectively focus on regions of interest. Existing models typically rely on top-down language information and learn …
Image CaptioningSelective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning
The human brain uses selective attention to filter perceptual input so that only the components that are useful for behaviour are processed using its limited computational resources. We focus on one particular form of vi…
Deep Reinforcement LearningMultiple-choiceOpen-Ended Question Answeringreinforcement-learning+2Attend Before you Act: Leveraging human visual attention for continual learning
When humans perform a task, such as playing a game, they selectively pay attention to certain parts of the visual input, gathering relevant information and sequentially combining it to build a representation from the sen…
Continual LearningDecision MakingTransfer Learning