ImageNot: A contrast with ImageNet preserves model rankings
We introduce ImageNot, a dataset designed to match the scale of ImageNet while differing drastically in other aspects. We show that key model architectures developed for ImageNet over the years rank identically when trained and evaluated on ImageNot to how they rank on ImageNet. This is true when training models from scratch or fine-tuning them. Moreover, the relative improvements of each model over earlier models strongly correlate in both datasets. We further give evidence that ImageNot has a similar utility as ImageNet for transfer learning purposes. Our work demonstrates a surprising degree of external validity in the relative performance of image classification models. This stands in contrast with absolute accuracy numbers that typically drop sharply even under small changes to a dataset.
Code (1)
Tasks
image-classificationImage ClassificationTransfer LearningSimilar Papers 제목 키워드 기반
Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend
Fr\'echet Inception Distance (FID) is a widely used metric for assessing synthetic image quality. It relies on an ImageNet-based feature extractor, making its applicability to medical imaging unclear. A recent trend is t…
Data AugmentationMedical Image GenerationAnatomy-Aware Contrastive Representation Learning for Fetal Ultrasound
Self-supervised contrastive representation learning offers the advantage of learning meaningful visual representations from unlabeled medical datasets for transfer learning. However, applying current contrastive learning…
AnatomyContrastive LearningRepresentation LearningTransfer LearningChronological Contrastive Learning: Few-Shot Progression Assessment in Irreversible Diseases
Quantitative disease severity scoring in medical imaging is costly, time-consuming, and subject to inter-reader variability. At the same time, clinical archives contain far more longitudinal imaging data than expert-anno…
Contrastive LearningFew-Shot LearningDelta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs
Large language models (LLMs) show strong potential for neural architecture generation, yet existing approaches produce complete model implementations from scratch -- computationally expensive and yielding verbose code. W…
Neural Architecture SearchCode GenerationA consensus set for the aggregation of partial rankings: the case of the Optimal Set of Bucket Orders Problem
In rank aggregation problems (RAP), the solution is usually a consensus ranking that generalizes a set of input orderings. There are different variants that differ not only in terms of the type of rankings that are used …
Subgroup Discovery