Unveiling Invariances via Neural Network Pruning
Invariance describes transformations that do not alter data's underlying semantics. Neural networks that preserve natural invariance capture good inductive biases and achieve superior performance. Hence, modern networks are handcrafted to handle well-known invariances (ex. translations). We propose a framework to learn novel network architectures that capture data-dependent invariances via pruning. Our learned architectures consistently outperform dense neural networks on both vision and tabular datasets in both efficiency and effectiveness. We demonstrate our framework on multiple deep learning models across 3 vision and 40 tabular datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Network PruningSimilar Papers 제목 키워드 기반
Skeletonisation Scale-Spaces
The medial axis transform is a well-known tool for shape recognition. Instead of the object contour, it equivalently describes a binary object in terms of a skeleton containing all centres of maximal inscribed discs. Whi…
Learning Invariances in Neural Networks from Training Data
Invariances to translations have imbued convolutional neural networks with powerful generalization properties. However, we often do not know a priori what invariances are present in the data, or to what extent a model sh…
image-classificationImage ClassificationMolecular Property PredictionProperty PredictionLearning Invariances in Neural Networks
Invariances to translations have imbued convolutional neural networks with powerful generalization properties. However, we often do not know a priori what invariances are present in the data, or to what extent a model sh…
image-classificationImage ClassificationMolecular Property PredictionProperty Prediction+1Do Invariances in Deep Neural Networks Align with Human Perception?
An evaluation criterion for safe and trustworthy deep learning is how well the invariances captured by representations of deep neural networks (DNNs) are shared with humans. We identify challenges in measuring these inva…
Data AugmentationSelf-Supervised LearningCharacterizing the invariances of learning algorithms using category theory
Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathem…
regression