Supervision Adaptation Balancing In-distribution Generalization and Out-of-distribution Detection
The discrepancy between in-distribution (ID) and out-of-distribution (OOD) samples can lead to \textit{distributional vulnerability} in deep neural networks, which can subsequently lead to high-confidence predictions for OOD samples. This is mainly due to the absence of OOD samples during training, which fails to constrain the network properly. To tackle this issue, several state-of-the-art methods include adding extra OOD samples to training and assign them with manually-defined labels. However, this practice can introduce unreliable labeling, negatively affecting ID classification. The distributional vulnerability presents a critical challenge for non-IID deep learning, which aims for OOD-tolerant ID classification by balancing ID generalization and OOD detection. In this paper, we introduce a novel \textit{supervision adaptation} approach to generate adaptive supervision information for OOD samples, making them more compatible with ID samples. Firstly, we measure the dependency between ID samples and their labels using mutual information, revealing that the supervision information can be represented in terms of negative probabilities across all classes. Secondly, we investigate data correlations between ID and OOD samples by solving a series of binary regression problems, with the goal of refining the supervision information for more distinctly separable ID classes. Our extensive experiments on four advanced network architectures, two ID datasets, and eleven diversified OOD datasets demonstrate the efficacy of our supervision adaptation approach in improving both ID classification and OOD detection capabilities.
Code (0)
등록된 구현이 없습니다.
Tasks
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionSimilar Papers 제목 키워드 기반
Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing
As training datasets grow larger, we aspire to develop models that generalize well to any diverse test distribution, even if the latter deviates significantly from the training data. Various approaches like domain adapta…
Domain AdaptationDomain GeneralizationLearning to Adapt SFT Data for Better Reasoning Generalization
Large language models (LLMs) have achieved remarkable progress, with post-training playing a crucial role in enhancing their reasoning capabilities. Among post-training paradigms, supervised fine-tuning (SFT) is widely u…
Reinforcement LearningDenseTRF: Texture-Aware Unsupervised Representation Adaptation for Surgical Scene Dense Prediction
Dense prediction tasks in surgical computer vision, such as segmentation and surgical zone prediction, can provide valuable guidance for laparoscopic and robotic surgery. However, these models often suffer from distribut…
Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition
Few-shot adaptation for Vision-Language Models (VLMs) presents a dilemma: balancing in-distribution accuracy with out-of-distribution generalization. Recent research has utilized low-level concepts such as visual attribu…
AttributeDecision MakingOut-of-Distribution Generalizationparameter-efficient fine-tuningEstimating Generalization under Distribution Shifts via Domain-Invariant Representations
When machine learning models are deployed on a test distribution different from the training distribution, they can perform poorly, but overestimate their performance. In this work, we aim to better estimate a model's pe…
Domain AdaptationModel Selection