Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data
Collecting more diverse and representative training data is often touted as a remedy for the disparate performance of machine learning predictors across subpopulations. However, a precise framework for understanding how dataset properties like diversity affect learning outcomes is largely lacking. By casting data collection as part of the learning process, we demonstrate that diverse representation in training data is key not only to increasing subgroup performances, but also to achieving population level objectives. Our analysis and experiments describe how dataset compositions influence performance and provide constructive results for using trends in existing data, alongside domain knowledge, to help guide intentional, objective-aware dataset design.
Code (1)
Tasks
DiversitySimilar Papers 제목 키워드 기반
Representation Invariance and Allocation: When Subgroup Balance Matters
Unequal representation of demographic groups in training data poses challenges to model generalisation across populations. Standard practice assumes that balancing subgroup representation optimises performance. However, …
Dual representations for systemic risk measures
The financial crisis showed the importance of measuring, allocating and regulating systemic risk. Recently, the systemic risk measures that can be decomposed into an aggregation function and a scalar measure of risk, rec…
Groupwise Maximin Fair Allocation of Indivisible Goods
We study the problem of allocating indivisible goods among n agents in a fair manner. For this problem, maximin share (MMS) is a well-studied solution concept which provides a fairness threshold. Specifically, maximin sh…
FairnessModel-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach
The interpretation of feature importance in machine learning models is challenging when features are dependent. Permutation feature importance (PFI) ignores such dependencies, which can cause misleading interpretations d…
Feature Importance