Saliency-based Sequential Image Attention with Multiset Prediction

2017-11-14 · NeurIPS 2017 12 · Sean Welleck, Jialin Mao, Kyunghyun Cho, Zheng Zhang

Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.

📄 PDF Abstract BibTeX arXiv:1711.05165

Code (0)

등록된 구현이 없습니다.

Tasks

Classification General Classification image-classification Image Classification Multi-Label Image Classification Prediction reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Loss Functions for Multiset Prediction

2017-11-14 · ICLR 2018 1 · Sean Welleck, Zixin Yao, Yu Gai, Jialin Mao 외

We study the problem of multiset prediction. The goal of multiset prediction is to train a predictor that maps an input to a multiset consisting of multiple items. Unlike existing problems in supervised learning, such as…

Decision Making Prediction Reinforcement Learning Reinforcement Learning (RL)+1

Inferring Attention Shift Ranks of Objects for Image Saliency

2020-06-01 · CVPR 2020 6 · Avishek Siris, Jianbo Jiao, Gary K.L. Tam, Xianghua Xie 외

Psychology studies and behavioural observation show that humans shift their attention from one location to another when viewing an image of a complex scene. This is due to the limited capacity of the human visual system …

Saliency Ranking

TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction

2023-01-05 · Bahar Aydemir, Ludo Hoffstetter, Tong Zhang, Mathieu Salzmann 외

Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity. Howe…

Object Object Recognition Prediction Saliency Prediction

TempSAL - Uncovering Temporal Information for Deep Saliency Prediction

2023-01-01 · CVPR 2023 1 · Bahar Aydemir, Ludo Hoffstetter, Tong Zhang, Mathieu Salzmann 외

Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information such as scene context, semantic relationships, gaze direction, and object dissimilarity. H…

Object Object Recognition Prediction Saliency Detection+1

Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation

2021-11-23 · ICLR 2022 4 · Yan Zhang, David W. Zhang, Simon Lacoste-Julien, Gertjan J. Burghouts 외

Most set prediction models in deep learning use set-equivariant operations, but they actually operate on multisets. We show that set-equivariant functions cannot represent certain functions on multisets, so we introduce …

Prediction Property Prediction