Quantifying the Reliance of Black-Box Decision-Makers on Variables of Interest
This paper introduces a framework for measuring how much black-box decision-makers rely on variables of interest. The framework adapts a permutation-based measure of variable importance from the explainable machine learning literature. With an emphasis on applicability, I present some of the framework's theoretical and computational properties, explain how reliance computations have policy implications, and work through an illustrative example. In the empirical application to interruptions by Supreme Court Justices during oral argument, I find that the effect of gender is more muted compared to the existing literature's estimate; I then use this paper's framework to compare Justices' reliance on gender and alignment to their reliance on experience, which are incomparable using regression coefficients.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
AI assistance in decision-making has become popular, yet people's inappropriate reliance on AI often leads to unsatisfactory human-AI collaboration performance. In this paper, through three pre-registered, randomized hum…
Decision MakingUnderstanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on the contrary, suggest that they can increa…
Decision MakingBehavioral Causal Inference
When inferring the causal effect of one variable on another from correlational data, a common practice by professional researchers as well as lay decision makers is to control for some set of exogenous confounding variab…
Causal InferenceAugmented Fairness: An Interpretable Model Augmenting Decision-Makers' Fairness
We propose a model-agnostic approach for mitigating the prediction bias of a black-box decision-maker, and in particular, a human decision-maker. Our method detects in the feature space where the black-box decision-maker…
Active LearningFairnessDynamic Information Sub-Selection for Decision Support
We introduce Dynamic Information Sub-Selection (DISS), a novel framework of AI assistance designed to enhance the performance of black-box decision-makers by tailoring their information processing on a per-instance basis…
Language ModelingLanguage ModellingLarge Language Model