Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer
Dealing with severe class imbalance poses a major challenge for real-world applications, especially when the accurate classification and generalization of minority classes is of primary interest. In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets. While existing solutions mostly appeal to sampling or weighting adjustments to alleviate the pathological imbalance, or imposing inductive bias to prioritize non-spurious associations, we take novel perspectives to promote sample efficiency and model generalization based on the invariance principles of causality. Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions. Such causal assumption enables efficient knowledge transfer from the dominant classes to their under-represented counterparts, even if the respective feature distributions show apparent disparities. This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes. Our development is orthogonal to the existing extreme classification techniques thus can be seamlessly integrated. The utility of our proposal is validated with an extensive set of synthetic and real-world computer vision tasks against SOTA solutions.
Code (1)
Tasks
Inductive BiasTransfer LearningSimilar Papers 제목 키워드 기반
CLAF: Contrastive Learning with Augmented Features for Imbalanced Semi-Supervised Learning
Due to the advantages of leveraging unlabeled data and learning meaningful representations, semi-supervised learning and contrastive learning have been progressively combined to achieve better performances in popular app…
Contrastive Learningimage-classificationImage ClassificationAn Asymmetric Contrastive Loss for Handling Imbalanced Datasets
Contrastive learning is a representation learning method performed by contrasting a sample to other similar samples so that they are brought closely together, forming clusters in the feature space. The learning process i…
Contrastive LearningRepresentation LearningImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classification
Graph contrastive learning (GCL) has attracted a surge of attention due to its superior performance for learning node/graph representations without labels. However, in practice, the underlying class distribution of unlab…
ClassificationContrastive LearningNode ClassificationOnline ClusteringExploring Balanced Feature Spaces for Representation Learning
Existing self-supervised learning (SSL) methods are mostly applied for training representation models from artificially balanced datasets (e.g., ImageNet). It is unclear how well they will perform in the practical scenar…
Contrastive LearningLong-tail LearningRepresentation LearningSelf-Supervised LearningA Tale of Two Classes: Adapting Supervised Contrastive Learning to Binary Imbalanced Datasets
Supervised contrastive learning (SupCon) has proven to be a powerful alternative to the standard cross-entropy loss for classification of multi-class balanced datasets. However, it struggles to learn well-conditioned…
Contrastive LearningMedical Diagnosis