REM: Routing Entropy Minimization for Capsule Networks

2022-04-04 · Riccardo Renzulli, Enzo Tartaglione, Marco Grangetto

Capsule Networks ambition is to build an explainable and biologically-inspired neural network model. One of their main innovations relies on the routing mechanism which extracts a parse tree: its main purpose is to explicitly build relationships between capsules. However, their true potential in terms of explainability has not surfaced yet: these relationships are extremely heterogeneous and difficult to understand. This paper proposes REM, a technique which minimizes the entropy of the parse tree-like structure, improving its explainability. We accomplish this by driving the model parameters distribution towards low entropy configurations, using a pruning mechanism as a proxy. We also generate static parse trees with no performance loss, showing that, with REM, Capsule Networks build stronger relationships between capsules.

📄 PDF Abstract BibTeX arXiv:2204.01298

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

REM Random Ensemble Mixture (REM) is an easy to implement extension of DQN inspired by Dropout.…

Learning Compositional Structures for Deep Learning: Why Routing-by-agreement is Necessary

2020-10-04 · Sai Raam Venkatraman, Ankit Anand, S. Balasubramanian, R. Raghunatha Sarma

A formal description of the compositionality of neural networks is associated directly with the formal grammar-structure of the objects it seeks to represent. This formal grammar-structure specifies the kind of component…

Attention routing between capsules

2019-07-03 · Jaewoong Choi, Hyun Seo, Suii Im, Myungjoo Kang

In this paper, we propose a new capsule network architecture called Attention Routing CapsuleNet (AR CapsNet). We replace the dynamic routing and squash activation function of the capsule network with dynamic routing (Ca…

Decoder

Hierarchical Object-Centric Learning with Capsule Networks

2024-05-30 · Riccardo Renzulli

Capsule networks (CapsNets) were introduced to address convolutional neural networks limitations, learning object-centric representations that are more robust, pose-aware, and interpretable. They organize neurons into gr…

Computational Efficiency Lung Nodule Segmentation Object

Mamba Capsule Routing Towards Part-Whole Relational Camouflaged Object Detection

2024-10-05 · Dingwen Zhang, Liangbo Cheng, Yi Liu, Xinggang Wang 외

The part-whole relational property endowed by Capsule Networks (CapsNets) has been known successful for camouflaged object detection due to its segmentation integrity. However, the previous Expectation Maximization (EM) …

Mamba object-detection Object Detection

Capsule networks with non-iterative cluster routing

2021-09-19 · Zhihao Zhao, Samuel Cheng

Capsule networks use routing algorithms to flow information between consecutive layers. In the existing routing procedures, capsules produce predictions (termed votes) for capsules of the next layer. In a nutshell, the n…