Deep Probabilistic Supervision for Image Classification
Supervised training of deep neural networks for classification typically relies on hard targets, which promote overconfidence and can limit calibration, generalization, and robustness. Self-distillation methods aim to mitigate this by leveraging inter-class and sample-specific information present in the model's own predictions, but often remain dependent on hard targets without explicitly modeling predictive uncertainty. With this in mind, we propose Deep Probabilistic Supervision (DPS), a principled learning framework constructing sample-specific target distributions via statistical inference on the model's own predictions, remaining independent of hard targets after initialization. We show that DPS consistently yields higher test accuracy (e.g., +2.0% for DenseNet-264 on ImageNet) and significantly lower Expected Calibration Error (ECE) (-40% ResNet-50, CIFAR-100) than existing self-distillation methods. When combined with a contrastive loss, DPS achieves state-of-the-art robustness under label noise.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #1081 | Image Classification | ImageNet | Deep | Top 1 Accuracy: 2.0 |
Similar Papers 제목 키워드 기반
Self-supervised Learning of Geometrically Stable Features Through Probabilistic Introspection
Self-supervision can dramatically cut back the amount of manually-labelled data required to train deep neural networks. While self-supervision has usually been considered for tasks such as image classification, in this p…
image-classificationImage ClassificationSelf-Supervised LearningUnsupervised Landmark DetectionGeneral Supervision via Probabilistic Transformations
Different types of training data have led to numerous schemes for supervised classification. Current learning techniques are tailored to one specific scheme and cannot handle general ensembles of training data. This pape…
ClassificationGeneral ClassificationWeakly Supervised Lesion Localization With Probabilistic-CAM Pooling
Localizing thoracic diseases on chest X-ray plays a critical role in clinical practices such as diagnosis and treatment planning. However, current deep learning based approaches often require strong supervision, e.g. ann…
CLAREL: Classification via retrieval loss for zero-shot learning
We address the problem of learning fine-grained cross-modal representations. We propose an instance-based deep metric learning approach in joint visual and textual space. The key novelty of this paper is that it shows th…
ClassificationGeneral ClassificationGeneralized Zero-Shot LearningMetric Learning+3Probabilistic Model Distillation for Semantic Correspondence
Semantic correspondence is a fundamental problem in computer vision, which aims at establishing dense correspondences across images depicting different instances under the same category. This task is challenging due …
modelRepresentation LearningSemantic correspondence