Capsule Network based Contrastive Learning of Unsupervised Visual Representations
Capsule Networks have shown tremendous advancement in the past decade, outperforming the traditional CNNs in various task due to it's equivariant properties. With the use of vector I/O which provides information of both magnitude and direction of an object or it's part, there lies an enormous possibility of using Capsule Networks in unsupervised learning environment for visual representation tasks such as multi class image classification. In this paper, we propose Contrastive Capsule (CoCa) Model which is a Siamese style Capsule Network using Contrastive loss with our novel architecture, training and testing algorithm. We evaluate the model on unsupervised image classification CIFAR-10 dataset and achieve a top-1 test accuracy of 70.50% and top-5 test accuracy of 98.10%. Due to our efficient architecture our model has 31 times less parameters and 71 times less FLOPs than the current SOTA in both supervised and unsupervised learning.
Code (1)
Tasks
Contrastive Learningimage-classificationImage ClassificationUnsupervised Image ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images
The function of constructing the hierarchy of objects is important to the visual process of the human brain. Previous studies have successfully adopted capsule networks to decompose the digits and faces into parts in an …
Face RecognitionGeometric Capsule Autoencoders for 3D Point Clouds
We propose a method to learn object representations from 3D point clouds using bundles of geometrically interpretable hidden units, which we call geometric capsules. Each geometric capsule represents a visual entity, suc…
ObjectRetrievalHP-Capsule: Unsupervised Face Part Discovery by Hierarchical Parsing Capsule Network
Capsule networks are designed to present the objects by a set of parts and their relationships, which provide an insight into the procedure of visual perception. Although recent works have shown the success of capsule ne…
State representation learning with recurrent capsule networks
Unsupervised learning of compact and relevant state representations has been proved very useful at solving complex reinforcement learning tasks. In this paper, we propose a recurrent capsule network that learns such repr…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Representation LearningTraining capsules as a routing-weighted product of expert neurons
Capsules are the multidimensional analogue to scalar neurons in neural networks, and because they are multidimensional, much more complex routing schemes can be used to pass information forward through the network than w…