Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition
The visual system processes a scene using a sequence of selective glimpses, each driven by spatial and object-based attention. These glimpses reflect what is relevant to the ongoing task and are selected through recurrent processing and recognition of the objects in the scene. In contrast, most models treat attention selection and recognition as separate stages in a feedforward process. Here we show that using capsule networks to create an object-centric hidden representation in an encoder-decoder model with iterative glimpse attention yields effective integration of attention and recognition. We evaluate our model on three multi-object recognition tasks; highly overlapping digits, digits among distracting clutter and house numbers, and show that it learns to effectively move its glimpse window, recognize and reconstruct the objects, all with only the classification as supervision. Our work takes a step toward a general architecture for how to integrate recurrent object-centric representation into the planning of attentional glimpses.
Code (1)
Tasks
DecoderObjectObject RecognitionSimilar Papers 제목 키워드 기반
Canonical Capsules: Self-Supervised Capsules in Canonical Pose
We propose a self-supervised capsule architecture for 3D point clouds. We compute capsule decompositions of objects through permutation-equivariant attention, and self-supervise the process by training with pairs of rand…
3D Point Cloud ReconstructionGeneral ClassificationObjectPoint cloud reconstructionSentiment Analysis by Capsules
In this paper, we propose RNN-Capsule, a capsule model based on Recurrent Neural Network (RNN) for sentiment analysis. For a given problem, one capsule is built for each sentiment category e.g., ‘positive’ and ‘negative’…
AttributeSentiment AnalysisSentiment ClassificationSpecificityLearning with Capsules: A Survey
Capsule networks were proposed as an alternative approach to Convolutional Neural Networks (CNNs) for learning object-centric representations, which can be leveraged for improved generalization and sample complexity. Unl…
Graph Representation LearningRepresentation LearningSurveyCapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing
In this work we propose a capsule-based approach for semi-supervised video object segmentation. Current video object segmentation methods are frame-based and often require optical flow to capture temporal consistency acr…
ObjectOne-shot visual object segmentationOptical Flow EstimationSegmentation+5Hierarchical Object-Centric Learning with Capsule Networks
Capsule networks (CapsNets) were introduced to address convolutional neural networks limitations, learning object-centric representations that are more robust, pose-aware, and interpretable. They organize neurons into gr…
Computational EfficiencyLung Nodule SegmentationObject