Long-tail learning via logit adjustment
Real-world classification problems typically exhibit an imbalanced or long-tailed label distribution, wherein many labels are associated with only a few samples. This poses a challenge for generalisation on such labels, and also makes na\"ive learning biased towards dominant labels. In this paper, we present two simple modifications of standard softmax cross-entropy training to cope with these challenges. Our techniques revisit the classic idea of logit adjustment based on the label frequencies, either applied post-hoc to a trained model, or enforced in the loss during training. Such adjustment encourages a large relative margin between logits of rare versus dominant labels. These techniques unify and generalise several recent proposals in the literature, while possessing firmer statistical grounding and empirical performance.
Code (3)
Tasks
Long-tail LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Local and Global Logit Adjustments for Long-Tailed Learning
Multi-expert ensemble models for long-tailed learning typically either learn diverse generalists from the whole dataset or aggregate specialists on different subsets. However, the former is insufficient for tail clas…
Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustment
Procedural activity videos often exhibit a long-tailed action distribution due to varying action frequencies and durations. However, state-of-the-art temporal action segmentation methods overlook the long tail and fail t…
Action SegmentationSegmentationTemporal Action SegmentationAdaptive Logit Adjustment Loss for Long-Tailed Visual Recognition
Data in the real world tends to exhibit a long-tailed label distribution, which poses great challenges for the training of neural networks in visual recognition. Existing methods tackle this problem mainly from the persp…
General ClassificationSemantic SimilaritySemantic Textual SimilarityLong-tailed Visual Recognition via Gaussian Clouded Logit Adjustment
Long-tailed data is still a big challenge for deep neural networks, even though they have achieved great success on balanced data. We observe that vanilla training on long-tailed data with cross-entropy loss makes the in…
Gradient-Aware Logit Adjustment Loss for Long-tailed Classifier
In the real-world setting, data often follows a long-tailed distribution, where head classes contain significantly more training samples than tail classes. Consequently, models trained on such data tend to be biased towa…