Invertible DenseNets
We introduce Invertible Dense Networks (i-DenseNets), a more parameter efficient alternative to Residual Flows. The method relies on an analysis of the Lipschitz continuity of the concatenation in DenseNets, where we enforce the invertibility of the network by satisfying the Lipschitz constraint. Additionally, we extend this method by proposing a learnable concatenation, which not only improves the model performance but also indicates the importance of the concatenated representation. We demonstrate the performance of i-DenseNets and Residual Flows on toy, MNIST, and CIFAR10 data. Both i-DenseNets outperform Residual Flows evaluated in negative log-likelihood, on all considered datasets under an equal parameter budget.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Invertible DenseNets with Concatenated LipSwish
We introduce Invertible Dense Networks (i-DenseNets), a more parameter efficient extension of Residual Flows. The method relies on an analysis of the Lipschitz continuity of the concatenation in DenseNets, where we enfor…
Density EstimationInvestigation of Densely Connected Convolutional Networks with Domain Adversarial Learning for Noise Robust Speech Recognition
We investigate densely connected convolutional networks (DenseNets) and their extension with domain adversarial training for noise robust speech recognition. DenseNets are very deep, compact convolutional neural networks…
Robust Speech Recognitionspeech-recognitionSpeech RecognitionDenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs
This paper revives Densely Connected Convolutional Networks (DenseNets) and reveals the underrated effectiveness over predominant ResNet-style architectures. We believe DenseNets' potential was overlooked due to untouche…
Fine-Grained Image ClassificationImage ClassificationInstance Segmentationobject-detection+2On Random Kernels of Residual Architectures
We derive finite width and depth corrections for the Neural Tangent Kernel (NTK) of ResNets and DenseNets. Our analysis reveals that finite size residual architectures are initialized much closer to the "kernel regime" t…
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
State-of-the-art approaches for semantic image segmentation are built on Convolutional Neural Networks (CNNs). The typical segmentation architecture is composed of (a) a downsampling path responsible for extracting coars…
image-classificationImage SegmentationSegmentationSemantic Segmentation