paper-with-me

홈 › Papers

Gradient Based Activations for Accurate Bias-Free Learning

2022-02-17 · Vinod K Kurmi, Rishabh Sharma, Yash Vardhan Sharma, Vinay P. Namboodiri

Bias mitigation in machine learning models is imperative, yet challenging. While several approaches have been proposed, one view towards mitigating bias is through adversarial learning. A discriminator is used to identify the bias attributes such as gender, age or race in question. This discriminator is used adversarially to ensure that it cannot distinguish the bias attributes. The main drawback in such a model is that it directly introduces a trade-off with accuracy as the features that the discriminator deems to be sensitive for discrimination of bias could be correlated with classification. In this work we solve the problem. We show that a biased discriminator can actually be used to improve this bias-accuracy tradeoff. Specifically, this is achieved by using a feature masking approach using the discriminator's gradients. We ensure that the features favoured for the bias discrimination are de-emphasized and the unbiased features are enhanced during classification. We show that this simple approach works well to reduce bias as well as improve accuracy significantly. We evaluate the proposed model on standard benchmarks. We improve the accuracy of the adversarial methods while maintaining or even improving the unbiasness and also outperform several other recent methods.

📄 PDF Abstract BibTeX arXiv:2202.10943

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks

2020-06-04 · NeurIPS 2020 12 · Alexander Shekhovtsov, Viktor Yanush, Boris Flach

In neural networks with binary activations and or binary weights the training by gradient descent is complicated as the model has piecewise constant response. We consider stochastic binary networks, obtained by adding no…

Towards Scalable Backpropagation-Free Gradient Estimation

2025-11-05 · Daniel Wang, Evan Markou, Dylan Campbell arxiv

While backpropagation--reverse-mode automatic differentiation--has been extraordinarily successful in deep learning, it requires two passes (forward and backward) through the neural network and the storage of intermediat…

Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositions

2026-05-27 · Thomas Vitry, Kieran Edgeworth, Stefan Wermter, Jae Hee Lee arxiv

Vision classifiers can exploit spurious correlations, achieving high in-distribution accuracy yet failing under distribution shift. Existing approaches to bias mitigation and analysis often depend on curated datasets, sp…

Minimum Variance Unbiased N:M Sparsity for the Neural Gradients

2022-03-21 · Brian Chmiel, Itay Hubara, Ron Banner, Daniel Soudry

In deep learning, fine-grained N:M sparsity reduces the data footprint and bandwidth of a General Matrix multiply (GEMM) up to x2, and doubles throughput by skipping computation of zero values. So far, it was mainly only…

Online Normalization for Training Neural Networks

2019-05-15 · NeurIPS 2019 12 · Vitaliy Chiley, Ilya Sharapov, Atli Kosson, Urs Koster 외

Online Normalization is a new technique for normalizing the hidden activations of a neural network. Like Batch Normalization, it normalizes the sample dimension. While Online Normalization does not use batches, it is as …

General Classificationimage-classificationImage ClassificationImage Segmentation+3