Improving Self-supervised Learning for Out-of-distribution Task via Auxiliary Classifier
In real world scenarios, out-of-distribution (OOD) datasets may have a large distributional shift from training datasets. This phenomena generally occurs when a trained classifier is deployed on varying dynamic environments, which causes a significant drop in performance. To tackle this issue, we are proposing an end-to-end deep multi-task network in this work. Observing a strong relationship between rotation prediction (self-supervised) accuracy and semantic classification accuracy on OOD tasks, we introduce an additional auxiliary classification head in our multi-task network along with semantic classification and rotation prediction head. To observe the influence of this addition classifier in improving the rotation prediction head, our proposed learning method is framed into bi-level optimisation problem where the upper-level is trained to update the parameters for semantic classification and rotation prediction head. In the lower-level optimisation, only the auxiliary classification head is updated through semantic classification head by fixing the parameters of the semantic classification head. The proposed method has been validated through three unseen OOD datasets where it exhibits a clear improvement in semantic classification accuracy than other two baseline methods. Our code is available on GitHub \url{https://github.com/harshita-555/OSSL}
Code (1)
Tasks
ClassificationPredictionSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Hierarchical Self-supervised Augmented Knowledge Distillation
Knowledge distillation often involves how to define and transfer knowledge from teacher to student effectively. Although recent self-supervised contrastive knowledge achieves the best performance, forcing the network to …
Knowledge DistillationRepresentation LearningTo Balance or Not to Balance: A Simple-yet-Effective Approach for Learning with Long-Tailed Distributions
Real-world visual data often exhibits a long-tailed distribution, where some ''head'' classes have a large number of samples, yet only a few samples are available for ''tail'' classes. Such imbalanced distribution causes…
Auxiliary LearningSelf-Supervised LearningKnowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution
Knowledge distillation (KD) is an effective framework that aims to transfer meaningful information from a large teacher to a smaller student. Generally, KD often involves how to define and transfer knowledge. Previous KD…
image-classificationImage ClassificationKnowledge Distillationobject-detection+2RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifier
State-of-the-art (SOTA) semi-supervised learning techniques, such as FixMatch and it's variants, have demonstrated impressive performance in classification tasks. However, these methods are not directly applicable to reg…
ClassificationregressionSemi-Supervised Audio RegressionSemi-Supervised Image Regression+1Automatic Self-supervised Learning for Social Recommendations
In recent years, researchers have attempted to exploit social relations to improve the performance in recommendation systems. Generally, most existing social recommendation methods heavily depends on substantial domain k…
Meta-LearningRecommendation SystemsRepresentation LearningSelf-Supervised Learning