Consistent Batch Normalization for Weighted Loss in Imbalanced-Data Environment
In this study, classification problems based on feedforward neural networks in a data-imbalanced environment are considered. Learning from an imbalanced dataset is one of the most important practical problems in the field of machine learning. A weighted loss function (WLF) based on a cost-sensitive approach is a well-known and effective method for imbalanced datasets. A combination of WLF and batch normalization (BN) is considered in this study. BN is considered as a powerful standard technique in the recent developments in deep learning. A simple combination of both methods leads to a size-inconsistency problem due to a mismatch between the interpretations of the effective size of the dataset in both methods. A simple modification to BN, called weighted BN (WBN), is proposed to correct the size mismatch. The idea of WBN is simple and natural. The proposed method in a data-imbalanced environment is validated using numerical experiments.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improvement of Batch Normalization in Imbalanced Data
In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine…
Disentangling Sampling from Training Budget in Class-Imbalanced CT Body Composition Segmentation
Class imbalance is a fundamental challenge in medical image segmentation, where frequent classes typically dominate training at the expense of rare classes. Loss-based approaches mitigate imbalance by reweighting the per…
Medical Image SegmentationFew-Shot LearningTowards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization
Test-Time Adaptation aims to adapt source domain model to testing data at inference stage with success demonstrated in adapting to unseen corruptions. However, these attempts may fail under more challenging real-world sc…
Test-time AdaptationA New Look at Ghost Normalization
Batch normalization (BatchNorm) is an effective yet poorly understood technique for neural network optimization. It is often assumed that the degradation in BatchNorm performance to smaller batch sizes stems from it havi…
Rethinking Normalization and Elimination Singularity in Neural Networks
In this paper, we study normalization methods for neural networks from the perspective of elimination singularity. Elimination singularities correspond to the points on the training trajectory where neurons become consis…
image-classificationImage ClassificationInstance Segmentationobject-detection+3