An Empirical Study of Modular Bias Mitigators and Ensembles
There are several bias mitigators that can reduce algorithmic bias in machine learning models but, unfortunately, the effect of mitigators on fairness is often not stable when measured across different data splits. A popular approach to train more stable models is ensemble learning. Ensembles, such as bagging, boosting, voting, or stacking, have been successful at making predictive performance more stable. One might therefore ask whether we can combine the advantages of bias mitigators and ensembles? To explore this question, we first need bias mitigators and ensembles to work together. We built an open-source library enabling the modular composition of 10 mitigators, 4 ensembles, and their corresponding hyperparameters. Based on this library, we empirically explored the space of combinations on 13 datasets, including datasets commonly used in fairness literature plus datasets newly curated by our library. Furthermore, we distilled the results into a guidance diagram for practitioners. We hope this paper will contribute towards improving stability in bias mitigation.
Code (0)
등록된 구현이 없습니다.
Tasks
Ensemble LearningFairnessSimilar Papers 제목 키워드 기반
Navigating Ensemble Configurations for Algorithmic Fairness
Bias mitigators can improve algorithmic fairness in machine learning models, but their effect on fairness is often not stable across data splits. A popular approach to train more stable models is ensemble learning, but u…
Ensemble LearningFairnessNavigateUncovering Bias in Face Generation Models
Recent advancements in GANs and diffusion models have enabled the creation of high-resolution, hyper-realistic images. However, these models may misrepresent certain social groups and present bias. Understanding bias in …
AttributeDecision MakingFace GenerationFairnessRandomization Can Reduce Both Bias and Variance: A Case Study in Random Forests
We study the often overlooked phenomenon, first noted in \cite{breiman2001random}, that random forests appear to reduce bias compared to bagging. Motivated by an interesting paper by \cite{mentch2020randomization}, where…
Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel
Identifying unfamiliar inputs, also known as out-of-distribution (OOD) detection, is a crucial property of any decision making process. A simple and empirically validated technique is based on deep ensembles where the va…
Decision MakingInductive BiasOut of Distribution (OOD) DetectionEnsembling over Classifiers: a Bias-Variance Perspective
Ensembles are a straightforward, remarkably effective method for improving the accuracy,calibration, and robustness of models on classification tasks; yet, the reasons that underlie their success remain an active area of…