Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence
This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as $f$-divergences and $\alpha$-R\'enyi divergences. Inspired by the theoretical foundations rooted in divergences, i.e., $f$-divergences and $\alpha$-R\'enyi divergence, we also provide valuable insights to enhance the understanding of our empirical risk functions and regularization techniques. In the pseudo-labeling and entropy minimization techniques as self-training methods for effective semi-supervised learning, the self-training process has some inherent mismatch between the true label and pseudo-label (noisy pseudo-labels) and some of our empirical risk functions are robust, concerning noisy pseudo-labels. Under some conditions, our empirical risk functions demonstrate better performance when compared to traditional self-training methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Pseudo LabelSimilar Papers 제목 키워드 기반
Deep Bregman Divergence for Contrastive Learning of Visual Representations
Deep Bregman divergence measures divergence of data points using neural networks which is beyond Euclidean distance and capable of capturing divergence over distributions. In this paper, we propose deep Bregman divergenc…
Contrastive Learningobject-detectionObject DetectionSelf-Supervised LearningSemi-supervised anomaly detection algorithm based on KL divergence (SAD-KL)
The unlabeled data are generally assumed to be normal data in detecting abnormal data via semisupervised learning. This assumption, however, causes inevitable detection error when distribution of unlabeled data is differ…
Anomaly DetectionSemi-supervised Anomaly DetectionSupervised Anomaly DetectionDiM: $f$-Divergence Minimization Guided Sharpness-Aware Optimization for Semi-supervised Medical Image Segmentation
As a technique to alleviate the pressure of data annotation, semi-supervised learning (SSL) has attracted widespread attention. In the specific domain of medical image segmentation, semi-supervised methods (SSMIS) have b…
Image SegmentationMedical Image SegmentationSemantic SegmentationSemi-supervised Medical Image SegmentationDeep Divergence Learning
Classical linear metric learning methods have recently been extended along two distinct lines: deep metric learning methods for learning embeddings of the data using neural networks, and Bregman divergence learning appro…
ClusteringMetric LearningMONET: Multiview Semi-supervised Keypoint Detection via Epipolar Divergence
This paper presents MONET -- an end-to-end semi-supervised learning framework for a keypoint detector using multiview image streams. In particular, we consider general subjects such as non-human species where attaining a…
Data AugmentationKeypoint Detection