Assessing External Validity Over Worst-case Subpopulations
Study populations are typically sampled from limited points in space and time, and marginalized groups are underrepresented. To assess the external validity of randomized and observational studies, we propose and evaluate the worst-case treatment effect (WTE) across all subpopulations of a given size, which guarantees positive findings remain valid over subpopulations. We develop a semiparametrically efficient estimator for the WTE that analyzes the external validity of the augmented inverse propensity weighted estimator for the average treatment effect. Our cross-fitting procedure leverages flexible nonparametric and machine learning-based estimates of nuisance parameters and is a regular root-$n$ estimator even when nuisance estimates converge more slowly. On real examples where external validity is of core concern, our proposed framework guards against brittle findings that are invalidated by unanticipated population shifts.
Code (1)
Tasks
Causal InferencevalidSimilar Papers 제목 키워드 기반
Distribution Learning with Valid Outputs Beyond the Worst-Case
Generative models at times produce "invalid" outputs, such as images with generation artifacts and unnatural sounds. Validity-constrained distribution learning attempts to address this problem by requiring that the learn…
validExternally Valid Selection of Experimental Sites via the k-Median Problem
We present a decision-theoretic justification for viewing the question of how to best choose where to experiment in order to optimize external validity as a k-median (clustering) problem, a popular problem in computer sc…
validNormalised clustering accuracy: An asymmetric external cluster validity measure
There is no, nor will there ever be, single best clustering algorithm. Nevertheless, we would still like to be able to distinguish between methods that work well on certain task types and those that systematically underp…
Clusteringset matchingA Review of Generalizability and Transportability
When assessing causal effects, determining the target population to which the results are intended to generalize is a critical decision. Randomized and observational studies each have strengths and limitations for estima…
Causal InferenceA Thermodynamic Picture of Financial Market and Model Risk
By treating the financial market as a thermodynamic system, we establish a one-to-one correspondence between thermodynamic variables and economic quantities. Measured by the expected loss under the worst-case scenario, f…