How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?
Out-of-distribution (OOD) detection is a critical task for reliable machine learning. Recent advances in representation learning give rise to distance-based OOD detection, where testing samples are detected as OOD if they are relatively far away from the centroids or prototypes of in-distribution (ID) classes. However, prior methods directly take off-the-shelf contrastive losses that suffice for classifying ID samples, but are not optimally designed when test inputs contain OOD samples. In this work, we propose CIDER, a novel representation learning framework that exploits hyperspherical embeddings for OOD detection. CIDER jointly optimizes two losses to promote strong ID-OOD separability: a dispersion loss that promotes large angular distances among different class prototypes, and a compactness loss that encourages samples to be close to their class prototypes. We analyze and establish the unexplored relationship between OOD detection performance and the embedding properties in the hyperspherical space, and demonstrate the importance of dispersion and compactness. CIDER establishes superior performance, outperforming the latest rival by 19.36% in FPR95. Code is available at https://github.com/deeplearning-wisc/cider.
Code (1)
Tasks
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionRepresentation LearningSimilar Papers 제목 키워드 기반
Learning Hyperspherical Time-Frequency Representations for Time-Series Out-of-Distribution Detection
Out-of-distribution (OOD) detection for time-series data remains comparatively underexplored compared to vision and language, with a limited principled understanding of how supervised time-series representations can be l…
Out-of-Distribution DetectionRepresentation LearningContrastive LearningHyperspherical Query Likelihood Models with Word Embeddings
This paper presents an initial study on hyperspherical query likelihood models (QLMs) for information retrieval (IR). Our motivation is to naturally utilize pre-trained word embeddings for probabilistic IR. To this end, …
Information RetrievalLanguage ModelingLanguage ModellingRetrieval+2Hyperspherical Embedding for Point Cloud Completion
Most real-world 3D measurements from depth sensors are incomplete, and to address this issue the point cloud completion task aims to predict the complete shapes of objects from partial observations. Previous works often …
DecoderMulti-Task LearningPoint Cloud CompletionYour Classifier Can Be Secretly a Likelihood-Based OOD Detector
The ability to detect out-of-distribution (OOD) inputs is critical to guarantee the reliability of classification models deployed in an open environment. A fundamental challenge in OOD detection is that a discriminative …
Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling
This paper introduces von Mises-Fisher exploration (vMF-exp), a scalable method for exploring large action sets in reinforcement learning problems where hyperspherical embedding vectors represent these actions. vMF-exp i…
Reinforcement Learning