Reducing the dilution: An analysis of the information sensitiveness of capsule network with a practical improvement method
Capsule network has shown various advantages over convolutional neural network (CNN). It keeps more precise spatial information than CNN and uses equivariance instead of invariance during inference and highly potential to be a new effective tool for visual tasks. However, the current capsule networks have incompatible performance with CNN when facing datasets with background and complex target objects and are lacking in universal and efficient regularization method. We analyze a main reason of the incompatible performance as the conflict between information sensitiveness of capsule network and unreasonably higher activation value distribution of capsules in primary capsule layer. Correspondingly, we propose a practical improvement method by restraining the activation value of capsules in primary capsule layer to suppress non-informative capsules and highlight discriminative capsules. In the experiments, the method has achieved better performances on various mainstream datasets. In addition, the proposed improvement methods can be seen as a suitable, simple and efficient regularization method that can be generally used in capsule network.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SBPF: Sensitiveness Based Pruning Framework For Convolutional Neural Network On Image Classification
Pruning techniques are used comprehensively to compress convolutional neural networks (CNNs) on image classification. However, the majority of pruning methods require a well pre-trained model to provide useful supporting…
image-classificationImage ClassificationR-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales. However, CoT's verbosity increases latency and memory usage and may propagate ea…
Taxonomy and Analysis of Sensitive User Queries in Generative AI Search
Although there has been a growing interest among industries in integrating generative LLMs into their services, limited experience and scarcity of resources act as a barrier in launching and servicing large-scale LLM-bas…
IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning
Capsule networks (CapsNets) are superior at modeling hierarchical spatial relationships but suffer from two critical limitations: high computational cost due to iterative dynamic routing and poor robustness under input c…
Representation LearningEvaluation of the Penetration Process of Fluorescent Collagenase Nanocapsules in a 3D Collagen Gel
One of the major limitations of nanomedicine is the scarce penetration of nanoparticles in tumoral tissues. These constrains have been tried to be solved by different strategies, such as the employ of polyethyleneglycol …