An Axiomatic Approach to Comparing Sensitivity Parameters
Many methods are available for assessing the importance of omitted variables. These methods typically make different, non-falsifiable assumptions. Hence the data alone cannot tell us which method is most appropriate. Since it is unreasonable to expect results to be robust against all possible robustness checks, researchers often use methods deemed "interpretable", a subjective criterion with no formal definition. In contrast, we develop the first formal, axiomatic framework for comparing and selecting among these methods. Our framework is analogous to the standard approach for comparing estimators based on their sampling distributions. We propose that sensitivity parameters be selected based on their covariate sampling distributions, a design distribution of parameter values induced by an assumption on how covariates are assigned to be observed or unobserved. Using this idea, we define a new concept of parameter consistency, and argue that a reasonable sensitivity parameter should be consistent. We prove that the literature's most popular approach is inconsistent, while several alternatives are consistent.
Code (0)
등록된 구현이 없습니다.
Tasks
SensitivitySimilar Papers 제목 키워드 기반
Differential Voting: Loss Functions For Axiomatically Diverse Aggregation of Heterogeneous Preferences
Reinforcement learning from human feedback (RLHF) implicitly aggregates heterogeneous human preferences into a single utility function, even though the underlying utilities of the participants are in practice diverse. He…
Reinforcement LearningAxiomatic Preference Modeling for Longform Question Answering
The remarkable abilities of large language models (LLMs) like GPT-4 partially stem from post-training processes like Reinforcement Learning from Human Feedback (RLHF) involving human preferences encoded in a reward model…
Question AnsweringThe Emergence of Relevance Through Axiomatic Attention Patterns During LoRA Fine-Tuning
LoRA fine-tuning is standard for adapting LLMs to reranking, but it remains unclear where in the network task-specific relevance behavior is learned and what attention-level changes accompany that learning. Through ablat…
Assessing Omitted Variable Bias when the Controls are Endogenous
Omitted variables are one of the most important threats to the identification of causal effects. Several widely used methods assess the impact of omitted variables on empirical conclusions by comparing measures of select…
SensitivityGlobal Sensitive-Based Input Shaping for UAV-Payload Precision Motion Control
This work presents a comprehensive analysis and design of global sensitivity-based input shapers for a 3D Unmanned Aerial Vehicle-payload system, emphasizing robustness against uncertainties in payload mass and rope leng…