Batch-Adaptive Annotations for Causal Inference with Complex-Embedded Outcomes
Estimating the causal effects of an intervention on outcomes is crucial. But often in domains such as healthcare and social services, this critical information about outcomes is documented by unstructured text, e.g. clinical notes in healthcare or case notes in social services. For example, street outreach to homeless populations is a common social services intervention, with ambiguous and hard-to-measure outcomes. Outreach workers compile case note records which are informative of outcomes. Although experts can succinctly extract relevant information from such unstructured case notes, it is costly or infeasible to do so for an entire corpus, which can span millions of notes. Recent advances in large language models (LLMs) enable scalable but potentially inaccurate annotation of unstructured text data. We leverage the decision of which datapoints should receive expert annotation vs. noisy imputation under budget constraints in a "design-based" estimator combining limited expert and plentiful noisy imputation data via \textit{causal inference with missing outcomes}. We develop a two-stage adaptive algorithm that optimizes the expert annotation probabilities, estimating the ATE with optimal asymptotic variance. We demonstrate how expert labels and LLM annotations can be combined strategically, efficiently and responsibly in a causal estimator. We run experiments on simulated data and two real-world datasets, including one on street outreach, to show the versatility of our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Causal InferenceImputationSimilar Papers 제목 키워드 기반
Inference for Batched Adaptive Experiments
The advantages of adaptive experiments have led to their rapid adoption in economics, other fields, as well as among practitioners. However, adaptive experiments pose challenges for causal inference. This note suggests a…
Causal InferenceOptimistic Algorithms for Adaptive Estimation of the Average Treatment Effect
Estimation and inference for the Average Treatment Effect (ATE) is a cornerstone of causal inference and often serves as the foundation for developing procedures for more complicated settings. Although traditionally anal…
Causal InferenceDARTS: Targeting Prognostic Covariates in Budget-Constrained Sequential Experiments
Randomized controlled trials typically assume that prognostic covariates are known and available at no cost. In practice, obtaining high-dimensional pretreatment data is costly, forcing a trade-off between covariate-adap…
Causal InferenceClimateCause: Complex and Implicit Causal Structures in Climate Reports
Understanding climate change requires reasoning over complex causal networks. Yet, existing causal discovery datasets predominantly capture explicit, direct causal relations. We introduce ClimateCause, a manually expert-…
Batch-wise Adaptive Pruning: Periodic Neuron Activation-Aware Weight Pruning for Language Reasoning Model
Large Reasoning Models (LRMs) achieve strong performance on complex tasks through extended chain-of-thought generation, but incur substantial computational costs during inference. In production settings, batched inferenc…