Out of Distribution Detection on ImageNet-O
Out of distribution (OOD) detection is a crucial part of making machine learning systems robust. The ImageNet-O dataset is an important tool in testing the robustness of ImageNet trained deep neural networks that are widely used across a variety of systems and applications. We aim to perform a comparative analysis of OOD detection methods on ImageNet-O, a first of its kind dataset with a label distribution different than that of ImageNet, that has been created to aid research in OOD detection for ImageNet models. As this dataset is fairly new, we aim to provide a comprehensive benchmarking of some of the current state of the art OOD detection methods on this novel dataset. This benchmarking covers a variety of model architectures, settings where we haves prior access to the OOD data versus when we don't, predictive score based approaches, deep generative approaches to OOD detection, and more.
Code (1)
Tasks
BenchmarkingOut-of-Distribution DetectionOut of Distribution (OOD) DetectionSimilar Papers 제목 키워드 기반
A framework for benchmarking class-out-of-distribution detection and its application to ImageNet
When deployed for risk-sensitive tasks, deep neural networks must be able to detect instances with labels from outside the distribution for which they were trained. In this paper we present a novel framework to benchmark…
BenchmarkingKnowledge DistillationOut-of-Distribution DetectionOut of Distribution (OOD) DetectionNatural Adversarial Examples
We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade. The datasets are collected with a simple adversarial filtration technique to create datasets with lim…
Adversarial AttackData AugmentationDomain GeneralizationOut-of-Distribution DetectionUnexplored Faces of Robustness and Out-of-Distribution: Covariate Shifts in Environment and Sensor Domains
Computer vision applications predict on digital images acquired by a camera from physical scenes through light. However, conventional robustness benchmarks rely on perturbations in digitized images, diverging from distri…
Out of Distribution (OOD) DetectionA Closer Look at In-Distribution vs. Out-of-Distribution Accuracy for Open-Set Test-time Adaptation
Open-set test-time adaptation (TTA) updates models on new data in the presence of input shifts and unknown output classes. While recent methods have made progress on improving in-distribution (InD) accuracy for known cla…
Test-time AdaptationIn or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation
Out-of-distribution (OOD) detection is the problem of identifying inputs which are unrelated to the in-distribution task. The OOD detection performance when the in-distribution (ID) is ImageNet-1K is commonly being teste…
Open Set LearningOut-of-Distribution DetectionOut of Distribution (OOD) Detection