A Strong Inductive Bias: Gzip for binary image classification
Deep learning networks have become the de-facto standard in Computer Vision for industry and research. However, recent developments in their cousin, Natural Language Processing (NLP), have shown that there are areas where parameter-less models with strong inductive biases can serve as computationally cheaper and simpler alternatives. We propose such a model for binary image classification: a nearest neighbor classifier combined with a general purpose compressor like Gzip. We test and compare it against popular deep learning networks like Resnet, EfficientNet and Mobilenet and show that it achieves better accuracy and utilizes significantly less space, more than two order of magnitude, within a few-shot setting. As a result, we believe that this underlines the untapped potential of models with stronger inductive biases in few-shot scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Learningimage-classificationImage ClassificationInductive BiasMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Feed-Forward Neural Networks Need Inductive Bias to Learn Equality Relations
Basic binary relations such as equality and inequality are fundamental to relational data structures. Neural networks should learn such relations and generalise to new unseen data. We show in this study, however, that th…
Inductive BiasRelational ReasoningYou Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differences
Progress in AI has largely been driven by methods that assume less. As compute and data increase, approaches with weaker inductive biases generally outperform those with stronger assumptions. This is particularly charact…
Self-Supervised LearningRepresentation LearningTowards Flexible Inductive Bias via Progressive Reparameterization Scheduling
There are two de facto standard architectures in recent computer vision: Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Strong inductive biases of convolutions help the model learn sample effectivel…
Inductive BiasSchedulingLearning high-level visual representations from a child's perspective without strong inductive biases
Young children develop sophisticated internal models of the world based on their visual experience. Can such models be learned from a child's visual experience without strong inductive biases? To investigate this, we tra…
ObjectObject LocalizationSelf-Supervised LearningThe 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs
We present a simple baseline for directly estimating the relative pose (rotation and translation, including scale) between two images. Deep methods have recently shown strong progress but often require complex or multi-s…
Inductive BiasPose PredictionTranslation