Connections Between Pairs of Filters Improve the Accuracy of Convolutional Neural Networks
While researchers continue to find new and improved network structures for CNNs, most of the newly invented architectures still rely on the traditional pattern of stacking convolutional blocks and separating them with pointwise activation functions. However, there are drawbacks to a network purely building on pointwise nonlinearities. One alternative is to introduce a pairwise connection between two filters of a network. Typical connection functions use multiplications or the minimum operation to realize logical AND connections. In this paper, we go one step further by demonstrating that CNNs can benefit from more general connections, which include parameters that are learned. With such parameters, the network is able to implement different connections in different network layers and better adapt the connection function to the task at hand.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
DiracNets: Training Very Deep Neural Networks Without Skip-Connections
Deep neural networks with skip-connections, such as ResNet, show excellent performance in various image classification benchmarks. It is though observed that the initial motivation behind them - training deeper networks …
image-classificationImage ClassificationDeep Expander Networks: Efficient Deep Networks from Graph Theory
Efficient CNN designs like ResNets and DenseNet were proposed to improve accuracy vs efficiency trade-offs. They essentially increased the connectivity, allowing efficient information flow across layers. Inspired by thes…
The 'tree-dependent components' of natural scenes are edge filters
We propose a new model for natural image statistics. Instead of minimizing dependency between components of natural images, we maximize a simple form of dependency in the form of tree-dependency. By learning filters and …
FormHeroFilter: Adaptive Spectral Graph Filter for Varying Heterophilic Relations
Graph heterophily, where connected nodes have different labels, has attracted significant interest recently. Most existing works adopt a simplified approach - using low-pass filters for homophilic graphs and high-pass fi…
Natural QuestionsError Feedback Approach for Quantization Noise Reduction of Distributed Graph Filters
This work introduces an error feedback approach for reducing quantization noise of distributed graph filters. It comes from error spectrum shaping techniques from state-space digital filters, and therefore establishes co…
Quantization